1[/==============================================================================
2    Copyright (C) 2001-2011 Hartmut Kaiser
3    Copyright (C) 2001-2011 Joel de Guzman
4    Copyright (C) 2001-2002 Daniel C. Nuffer
5
6    Distributed under the Boost Software License, Version 1.0. (See accompanying
7    file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
8===============================================================================/]
9
10[section:multi_pass The multi pass iterator]
11
12Backtracking in __qi__ requires the use of the following types of iterator:
13forward, bidirectional, or random access. Because of backtracking, input
14iterators cannot be used. Therefore, the standard library classes
15`std::istreambuf_iterator` and `std::istream_iterator`, that fall under the
16category of input iterators, cannot be used. Another input iterator that is of
17interest is one that wraps a lexer, such as LEX.
18
19[note In general, __qi__ generates recursive descent parser which require
20      backtracking parsers by design. For this reason we need to provide at
21      least forward iterators to any of __qi__'s API functions. This is not an
22      absolute requirement though. In the future, we shall see more
23      deterministic parsers that require no more than 1 character (token) of
24      lookahead. Such parsers allow us to use input iterators such as the
25      `std::istream_iterator` as is. ]
26
27Backtracking can be implemented only if we are allowed to save an iterator
28position, i.e. making a copy of the current iterator. Unfortunately, with an
29input iterator, there is no way to do so, and thus input iterators will not
30work with backtracking in __qi__. One solution to this problem is to simply
31load all the data to be parsed into a container, such as a vector or deque,
32and then pass the begin and end of the container to __qi__. This method can be
33too memory intensive for certain applications, which is why the `multi_pass`
34iterator was created.
35
36[heading Using the multi_pass]
37
38The `multi_pass` iterator will convert any input iterator into a forward
39iterator suitable for use with __qi__. `multi_pass` will buffer data when
40needed and will discard the buffer when its contents is not needed anymore.
41This happens either if only one copy of the iterator exists or if no
42backtracking can occur.
43
44A grammar must be designed with care if the `multi_pass` iterator is used.
45Any rule that may need to backtrack, such as one that contains an alternative,
46will cause data to be buffered. The rules that are optimal to use are
47repetition constructs (as kleene and plus).
48
49Sequences of the form `a >> b` will buffer data as well. This is different from
50the behavior of __classic__ but for a good reason. Sequences need to reset the
51current iterator to its initial state if one of the components of a sequence
52fails to match. To compensate for this behavior we added functionality to
53the `expect` parsers (i.e. constructs like `a > b`). Expectation points introduce
54deterministic points into the grammar ensuring no backtracking can occur if
55they match. For this reason we clear the buffers of any multi_pass iterator
56on each expectation point, ensuring minimal buffer content even for large
57grammars.
58
59[important  If you use an error handler in conjunction with the `expect` parser
60            while utilizing a `multi_pass` iterator and you intend to use
61            the error handler to force a `retry` or a `fail` (see the
62            description of error handlers - __fixme__: insert link), then
63            you need to instantiate the error handler using `retry` or `fail`,
64            for instance:
65``
66        rule r<iterator_type> r;
67        on_error<retry>(r, std::cout << phoenix::val("Error!"));
68``
69            If you fail to do so the resulting code will trigger an assert
70            statement at runtime.]
71
72Any rule that repeats, such as kleene_star (`*a`) or positive such as (`+a`),
73will only buffer the data for the current repetition.
74
75In typical grammars, ambiguity and therefore lookahead is often localized. In
76fact, many well designed languages are fully deterministic and require no
77lookahead at all. Peeking at the first character from the input will
78immediately determine the alternative branch to take. Yet, even with highly
79ambiguous grammars, alternatives are often of the form `*(a | b | c | d)`.
80The input iterator moves on and is never stuck at the beginning. Let's look at
81a Pascal snippet for example:
82
83    program =
84            programHeading >> block >> '.'
85        ;
86
87    block =
88           *(   labelDeclarationPart
89            |   constantDefinitionPart
90            |   typeDefinitionPart
91            |   variableDeclarationPart
92            |   procedureAndFunctionDeclarationPart
93            )
94        >>  statementPart
95        ;
96
97Notice the alternatives inside the Kleene star in the rule block . The rule
98gobbles the input in a linear manner and throws away the past history with each
99iteration. As this is fully deterministic LL(1) grammar, each failed
100alternative only has to peek 1 character (token). The alternative that consumes
101more than 1 character (token) is definitely a winner. After which, the Kleene
102star moves on to the next.
103
104Now, after the lecture on the features to be careful with when using
105`multi_pass`, you may think that `multi_pass` is way too restrictive to use.
106That's not the case. If your grammar is deterministic, you can make use of
107the `flush_multi_pass` pseudo parser in your grammar to ensure that data is not
108buffered when unnecessary (`flush_multi_pass` is available from the __qi__
109parser __repo__).
110
111Here we present a minimal example showing a minimal use case. The `multi_pass`
112iterator is highly configurable, but the default policies have been chosen so
113that its easily usable with input iterators such as `std::istreambuf_iterator`.
114For the complete source code of this example please refer to
115[@../../example/support/multi_pass.cpp multi_pass.cpp].
116
117[import ../../example/support/multi_pass.cpp]
118[tutorial_multi_pass]
119
120[heading Using the flush_multi_pass parser]
121
122The __spirit__ __repo__ contains the `flush_multi_pass` parser component.
123This is usable in conjunction with the `multi_pass` iterator to minimize the
124buffering. It allows to insert explicit synchronization points into your
125grammar where it is safe to clear any stored input as it is ensured that no
126backtracking can occur at this point anymore.
127
128When the `flush_multi_pass` parser is used with `multi_pass`, it will call
129`multi_pass::clear_queue()`. This will cause any buffered data to be erased.
130This also will invalidate all other copies of multi_pass and they should not
131be used. If they are, an `boost::illegal_backtracking` exception will be
132thrown.
133
134[heading The multi_pass Policies]
135
136The `multi_pass` iterator is a templated class configurable using policies.
137The description of `multi_pass` above is how it was originally implemented
138(before it used policies), and is the default configuration now. But,
139`multi_pass` is capable of much more. Because of the open-ended nature of
140policies, you can write your own policy to make `multi_pass` behave in a way
141that we never before imagined.
142
143The multi_pass class has two template parameters:
144
145[variablelist The multi_pass template parameters
146    [[Input]    [The type multi_pass uses to acquire it's input. This is
147                 typically an input iterator, or functor.]]
148    [[Policies] [The combined policies to use to create an instance of a
149                 multi_pass iterator. This combined policy type is described
150                 below]]
151]
152
153It is possible to implement all of the required functionality of the combined
154policy in a single class. But it has shown to be more convenient to split this
155into four different groups of functions, i.e. four separate, but well
156coordinated policies. For this reason the `multi_pass` library
157implements a template `iterator_policies::default_policy` allowing to combine
158several different policies, each implementing one of the functionality groups:
159
160[table Policies needed for default_policy template
161    [[Template Parameter]   [Description]]
162    [[`OwnershipPolicy`]    [This policy determines how `multi_pass` deals with
163                             it's shared components.]]
164    [[`CheckingPolicy`]     [This policy determines how checking for invalid
165                             iterators is done.]]
166    [[`InputPolicy`]        [A class that defines how `multi_pass` acquires its
167                             input. The `InputPolicy` is parameterized by the
168                             `Input` template parameter to the `multi_pass`.]]
169    [[`StoragePolicy`]      [The buffering scheme used by `multi_pass` is
170                             determined and managed by the StoragePolicy.]]
171]
172
173The `multi_pass` library contains several predefined policy implementations
174for each of the policy types as described above. First we will describe those
175predefined types. Afterwards we will give some guidelines how you can write
176your own policy implementations.
177
178[heading Predefined policies]
179
180All predefined `multi_pass` policies are defined in the namespace
181`boost::spirit::iterator_policies`.
182
183[table Predefined policy classes
184    [[Class name]         [Description]]
185    [[*InputPolicy* classes]]
186    [[`input_iterator`]   [This policy directs `multi_pass` to read from an
187                           input iterator of type `Input`.]]
188    [[`buffering_input_iterator`]   [This policy directs `multi_pass` to read from an
189                           input iterator of type `Input`. Additionally it buffers
190                           the last character received from the underlying iterator.
191                           This allows to wrap iterators not buffering the last
192                           character on their own (as `std::istreambuf_iterator`).]]
193    [[`istream`]          [This policy directs `multi_pass` to read from an
194                           input stream of type `Input` (usually a
195                           `std::basic_istream`).]]
196    [[`lex_input`]        [This policy obtains it's input by calling yylex(),
197                           which would typically be provided by a scanner
198                           generated by __flex__. If you use this policy your code
199                           must link against a __flex__ generated scanner.]]
200    [[`functor_input`]    [This input policy obtains it's data by calling a
201                           functor of type `Input`. The functor must meet
202                           certain requirements. It must have a typedef called
203                           `result_type` which should be the type returned
204                           from `operator()`. Also, since an input policy needs
205                           a way to determine when the end of input has been
206                           reached, the functor must contain a static variable
207                           named `eof` which is comparable to a variable of
208                           `result_type`.]]
209    [[`split_functor_input`][This is essentially the same as the `functor_input`
210                           policy except that the (user supplied) function
211                           object exposes separate `unique` and `shared` sub
212                           classes, allowing to integrate the functors /unique/
213                           data members with the `multi_pass` data items held
214                           by each instance and its /shared/ data members will
215                           be integrated with the `multi_pass` members shared
216                           by all copies.]]
217
218    [[*OwnershipPolicy* classes]]
219    [[`ref_counted`]      [This class uses a reference counting scheme.
220                           The `multi_pass` will delete it's shared components
221                           when the count reaches zero.]]
222    [[`first_owner`]      [When this policy is used, the first `multi_pass`
223                           created will be the one that deletes the shared data.
224                           Each copy will not take ownership of the shared data.
225                           This works well for __spirit__, since no dynamic
226                           allocation of iterators is done. All copies are made
227                           on the stack, so the original iterator has the
228                           longest lifespan.]]
229
230    [[*CheckingPolicy* classes]]
231    [[`no_check`]         [This policy does no checking at all.]]
232    [[`buf_id_check`]     [This policy keeps around a buffer id, or a buffer
233                           age. Every time `clear_queue()` is called on a
234                           `multi_pass` iterator, it is possible that all other
235                           iterators become invalid. When `clear_queue()` is
236                           called, `buf_id_check` increments the buffer id.
237                           When an iterator is dereferenced, this policy checks
238                           that the buffer id of the iterator matches the shared
239                           buffer id. This policy is most effective when used
240                           together with the `split_std_deque` StoragePolicy.
241                           It should not be used with the `fixed_size_queue`
242                           StoragePolicy, because it will not detect iterator
243                           dereferences that are out of range.]]
244    [[full_check]         [This policy has not been implemented yet. When it
245                           is, it will keep track of all iterators and make
246                           sure that they are all valid. This will be mostly
247                           useful for debugging purposes as it will incur
248                           significant overhead.]]
249
250    [[*StoragePolicy* classes]]
251    [[`split_std_deque`]  [Despite its name this policy keeps all buffered data
252                           in a `std::vector`. All data is stored as long as
253                           there is more than one iterator. Once the iterator
254                           count goes down to one, and the queue is no longer
255                           needed, it is cleared, freeing up memory. The queue
256                           can also be forcibly cleared by calling
257                           `multi_pass::clear_queue()`.]]
258    [[`fixed_size_queue<N>`][This policy keeps a circular buffer that is size
259                            `N+1` and stores `N` elements. `fixed_size_queue`
260                            is a template with a `std::size_t` parameter that
261                            specified the queue size. It is your responsibility
262                            to ensure that `N` is big enough for your parser.
263                            Whenever the foremost iterator is incremented, the
264                            last character of the buffer is automatically
265                            erased. Currently there is no way to tell if an
266                            iterator is trailing too far behind and has become
267                            invalid. No dynamic allocation is done by this
268                            policy during normal iterator operation, only on
269                            initial construction. The memory usage of this
270                            `StoragePolicy` is set at `N+1` bytes, unlike
271                            `split_std_deque`, which is unbounded.]]
272]
273
274[heading Combinations: How to specify your own custom multi_pass]
275
276The beauty of policy based designs is that you can mix and match policies to
277create your own custom iterator by selecting the policies you want. Here's an
278example of how to specify a custom `multi_pass` that wraps an
279`std::istream_iterator<char>`, and is slightly more efficient than the default
280`multi_pass` (as generated by the `make_default_multi_pass()` API function)
281because it uses the `iterator_policies::first_owner` OwnershipPolicy and the
282`iterator_policies::no_check` CheckingPolicy:
283
284    typedef multi_pass<
285        std::istream_iterator<char>
286      , iterator_policies::default_policy<
287            iterator_policies::first_owner
288          , iterator_policies::no_check
289          , iterator_policies::buffering_input_iterator
290          , iterator_policies::split_std_deque
291        >
292    > first_owner_multi_pass_type;
293
294The default template parameters for `iterator_policies::default_policy` are:
295
296* `iterator_policies::ref_counted` OwnershipPolicy
297* `iterator_policies::no_check` CheckingPolicy, if `BOOST_SPIRIT_DEBUG` is
298  defined: `iterator_policies::buf_id_check` CheckingPolicy
299* `iterator_policies::buffering_input_iterator` InputPolicy, and
300* `iterator_policies::split_std_deque` StoragePolicy.
301
302So if you use `multi_pass<std::istream_iterator<char> >` you will get those
303pre-defined behaviors while wrapping an `std::istream_iterator<char>`.
304
305[heading Dealing with constant look ahead]
306
307There is one other pre-defined class called `look_ahead`. The class
308`look_ahead` is another predefine `multi_pass` iterator type. It has two
309template parameters: `Input`, the type of the input iterator to wrap, and a
310`std::size_t N`, which specifies the size of the buffer to the
311`fixed_size_queue` policy. While the default multi_pass configuration is
312designed for safety, `look_ahead` is designed for speed. `look_ahead` is derived
313from a multi_pass with the following policies: `input_iterator` InputPolicy,
314`first_owner` OwnershipPolicy, `no_check` CheckingPolicy, and
315`fixed_size_queue<N>` StoragePolicy.
316
317This iterator is defined by including the files:
318
319    // forwards to <boost/spirit/home/support/look_ahead.hpp>
320    #include <boost/spirit/include/support_look_ahead.hpp>
321
322Also, see __include_structure__.
323
324[heading Reading from standard input streams]
325
326Yet another predefined iterator for wrapping standard input streams (usually a
327`std::basic_istream<>`) is called `basic_istream_iterator<Char, Traits>`. This
328class is usable as a drop in replacement for `std::istream_iterator<Char, Traits>`.
329Its only difference is that it is a forward iterator (instead of the
330`std::istream_iterator`, which is an input iterator). `basic_istream_iterator`
331is derived from a multi_pass with the following policies: `istream` InputPolicy,
332`ref_counted` OwnershipPolicy, `no_check` CheckingPolicy, and
333`split_std_deque` StoragePolicy.
334
335There exists an additional predefined typedef:
336
337    typedef basic_istream_iterator<char, std::char_traits<char> > istream_iterator;
338
339This iterator is defined by including the files:
340
341    // forwards to <boost/spirit/home/support/istream_iterator.hpp>
342    #include <boost/spirit/include/support_istream_iterator.hpp>
343
344Also, see __include_structure__.
345
346[heading How to write a functor for use with the `functor_input` InputPolicy]
347
348If you want to use the `functor_input` InputPolicy, you can write your own
349function object that will supply the input to `multi_pass`. The function object
350must satisfy several requirements. It must have a typedef `result_type` which
351specifies the return type of its `operator()`. This is standard practice in the
352STL. Also, it must supply a static variable called eof which is compared against
353to know whether the input has reached the end. Last but not least the function
354object must be default constructible. Here is an example:
355
356    #include <iostream>
357    #include <boost/spirit/home/qi.hpp>
358    #include <boost/spirit/home/support.hpp>
359    #include <boost/spirit/home/support/multi_pass.hpp>
360    #include <boost/spirit/home/support/iterators/detail/functor_input_policy.hpp>
361
362    // define the function object
363    class iterate_a2m
364    {
365    public:
366        typedef char result_type;
367
368        iterate_a2m() : c_('A') {}
369        iterate_a2m(char c) : c_(c) {}
370
371        result_type operator()()
372        {
373            if (c_ == 'M')
374                return eof;
375            return c_++;
376        }
377
378        static result_type eof;
379
380    private:
381        char c_;
382    };
383
384    iterate_a2m::result_type iterate_a2m::eof = iterate_a2m::result_type('M');
385
386    using namespace boost::spirit;
387
388    // create two iterators using the define function object, one of which is
389    // an end iterator
390    typedef multi_pass<iterate_a2m
391      , iterator_policies::first_owner
392      , iterator_policies::no_check
393      , iterator_policies::functor_input
394      , iterator_policies::split_std_deque>
395    functor_multi_pass_type;
396
397    int main()
398    {
399        functor_multi_pass_type first = functor_multi_pass_type(iterate_a2m());
400        functor_multi_pass_type last;
401
402        // use the iterators: this will print "ABCDEFGHIJKL"
403        while (first != last) {
404            std::cout << *first;
405            ++first;
406        }
407        std::cout << std::endl;
408        return 0;
409    }
410
411[heading How to write policies for use with multi_pass]
412
413All policies to be used with the `default_policy` template need to have two
414embedded classes: `unique` and `shared`. The `unique` class needs to implement
415all required functions for a particular policy type. In addition it may hold
416all member data items being /unique/ for a particular instance of a `multi_pass`
417(hence the name). The `shared` class does not expose any member functions
418(except sometimes a constructor), but it may hold all member data items to be
419/shared/ between all copies of a particular `multi_pass`.
420
421[heading InputPolicy]
422
423An `InputPolicy` must have the following interface:
424
425    struct input_policy
426    {
427        // Input is the same type used as the first template parameter
428        // while instantiating the multi_pass
429        template <typename Input>
430        struct unique
431        {
432            // these typedef's will be exposed as the multi_pass iterator
433            // properties
434            typedef __unspecified_type__ value_type;
435            typedef __unspecified_type__ difference_type;
436            typedef __unspecified_type__ distance_type;
437            typedef __unspecified_type__ pointer;
438            typedef __unspecified_type__ reference;
439
440            unique() {}
441            explicit unique(Input) {}
442
443            // destroy is called whenever the last copy of a multi_pass is
444            // destructed (ownership_policy::release() returned true)
445            //
446            //   mp:    is a reference to the whole multi_pass instance
447            template <typename MultiPass>
448            static void destroy(MultiPass& mp);
449
450            // swap is called by multi_pass::swap()
451            void swap(unique&);
452
453            // get_input is called whenever the next input character/token
454            // should be fetched.
455            //
456            //   mp:    is a reference to the whole multi_pass instance
457            //
458            // This method is expected to return a reference to the next
459            // character/token
460            template <typename MultiPass>
461            static typename MultiPass::reference get_input(MultiPass& mp);
462
463            // advance_input is called whenever the underlying input stream
464            // should be advanced so that the next call to get_input will be
465            // able to return the next input character/token
466            //
467            //   mp:    is a reference to the whole multi_pass instance
468            template <typename MultiPass>
469            static void advance_input(MultiPass& mp);
470
471            // input_at_eof is called to test whether this instance is a
472            // end of input iterator.
473            //
474            //   mp:    is a reference to the whole multi_pass instance
475            //
476            // This method is expected to return true if the end of input is
477            // reached. It is often used in the implementation of the function
478            // storage_policy::is_eof.
479            template <typename MultiPass>
480            static bool input_at_eof(MultiPass const& mp);
481
482            // input_is_valid is called to verify if the parameter t represents
483            // a valid input character/token
484            //
485            //   mp:    is a reference to the whole multi_pass instance
486            //   t:     is the character/token to test for validity
487            //
488            // This method is expected to return true if the parameter t
489            // represents a valid character/token.
490            template <typename MultiPass>
491            static bool input_is_valid(MultiPass const& mp, value_type const& t);
492        };
493
494        // Input is the same type used as the first template parameter passed
495        // while instantiating the multi_pass
496        template <typename Input>
497        struct shared
498        {
499            explicit shared(Input) {}
500        };
501    };
502
503It is possible to derive the struct `unique` from the type
504`boost::spirit::detail::default_input_policy`. This type implements a minimal
505sufficient interface for some of the required functions, simplifying the task
506of writing a new input policy.
507
508This class may implement a function `destroy()` being called during destruction
509of the last copy of a `multi_pass`. This function should be used to free any of
510the shared data items the policy might have allocated during construction of
511its `shared` part. Because of the way `multi_pass` is implemented any allocated
512data members in `shared` should _not_ be deep copied in a copy constructor of
513`shared`.
514
515[heading OwnershipPolicy]
516
517The `OwnershipPolicy` must have the following interface:
518
519    struct ownership_policy
520    {
521        struct unique
522        {
523            // destroy is called whenever the last copy of a multi_pass is
524            // destructed (ownership_policy::release() returned true)
525            //
526            //   mp:    is a reference to the whole multi_pass instance
527            template <typename MultiPass>
528            static void destroy(MultiPass& mp);
529
530            // swap is called by multi_pass::swap()
531            void swap(unique&);
532
533            // clone is called whenever a multi_pass is copied
534            //
535            //   mp:    is a reference to the whole multi_pass instance
536            template <typename MultiPass>
537            static void clone(MultiPass& mp);
538
539            // release is called whenever a multi_pass is destroyed
540            //
541            //   mp:    is a reference to the whole multi_pass instance
542            //
543            // The method is expected to return true if the destructed
544            // instance is the last copy of a particular multi_pass.
545            template <typename MultiPass>
546            static bool release(MultiPass& mp);
547
548            // is_unique is called to test whether this instance is the only
549            // existing copy of a particular multi_pass
550            //
551            //   mp:    is a reference to the whole multi_pass instance
552            //
553            // The method is expected to return true if this instance is unique
554            // (no other copies of this multi_pass exist).
555            template <typename MultiPass>
556            static bool is_unique(MultiPass const& mp);
557        };
558
559        struct shared {};
560    };
561
562It is possible to derive the struct `unique` from the type
563`boost::spirit::detail::default_ownership_policy`. This type implements a
564minimal sufficient interface for some of the required functions, simplifying
565the task of writing a new ownership policy.
566
567This class may implement a function `destroy()` being called during destruction
568of the last copy of a `multi_pass`. This function should be used to free any of
569the shared data items the policy might have allocated during construction of
570its `shared` part. Because of the way `multi_pass` is implemented any allocated
571data members in `shared` should _not_ be deep copied in a copy constructor of
572`shared`.
573
574[heading CheckingPolicy]
575
576The `CheckingPolicy` must have the following interface:
577
578    struct checking_policy
579    {
580        struct unique
581        {
582            // swap is called by multi_pass::swap()
583            void swap(unique&);
584
585            // destroy is called whenever the last copy of a multi_pass is
586            // destructed (ownership_policy::release() returned true)
587            //
588            //   mp:    is a reference to the whole multi_pass instance
589            template <typename MultiPass>
590            static void destroy(MultiPass& mp);
591
592            // docheck is called before the multi_pass is dereferenced or
593            // incremented.
594            //
595            //   mp:    is a reference to the whole multi_pass instance
596            //
597            // This method is expected to make sure the multi_pass instance is
598            // still valid. If it is invalid an exception should be thrown.
599            template <typename MultiPass>
600            static void docheck(MultiPass const& mp);
601
602            // clear_queue is called whenever the function
603            // multi_pass::clear_queue is called on this instance
604            //
605            //   mp:    is a reference to the whole multi_pass instance
606            template <typename MultiPass>
607            static void clear_queue(MultiPass& mp);
608        };
609
610        struct shared {};
611    };
612
613It is possible to derive the struct `unique` from the type
614`boost::spirit::detail::default_checking_policy`. This type implements a
615minimal sufficient interface for some of the required functions, simplifying
616the task of writing a new checking policy.
617
618This class may implement a function `destroy()` being called during destruction
619of the last copy of a `multi_pass`. This function should be used to free any of
620the shared data items the policy might have allocated during construction of
621its `shared` part. Because of the way `multi_pass` is implemented any allocated
622data members in `shared` should _not_ be deep copied in a copy constructor of
623`shared`.
624
625[heading StoragePolicy]
626
627A `StoragePolicy` must have the following interface:
628
629    struct storage_policy
630    {
631        // Value is the same type as typename MultiPass::value_type
632        template <typename Value>
633        struct unique
634        {
635            // destroy is called whenever the last copy of a multi_pass is
636            // destructed (ownership_policy::release() returned true)
637            //
638            //   mp:    is a reference to the whole multi_pass instance
639            template <typename MultiPass>
640            static void destroy(MultiPass& mp);
641
642            // swap is called by multi_pass::swap()
643            void swap(unique&);
644
645            // dereference is called whenever multi_pass::operator*() is invoked
646            //
647            //   mp:    is a reference to the whole multi_pass instance
648            //
649            // This function is expected to return a reference to the current
650            // character/token.
651            template <typename MultiPass>
652            static typename MultiPass::reference dereference(MultiPass const& mp);
653
654            // increment is called whenever multi_pass::operator++ is invoked
655            //
656            //   mp:    is a reference to the whole multi_pass instance
657            template <typename MultiPass>
658            static void increment(MultiPass& mp);
659
660            //
661            //   mp:    is a reference to the whole multi_pass instance
662            template <typename MultiPass>
663            static void clear_queue(MultiPass& mp);
664
665            // is_eof is called to test whether this instance is a end of input
666            // iterator.
667            //
668            //   mp:    is a reference to the whole multi_pass instance
669            //
670            // This method is expected to return true if the end of input is
671            // reached.
672            template <typename MultiPass>
673            static bool is_eof(MultiPass const& mp);
674
675            // less_than is called whenever multi_pass::operator==() is invoked
676            //
677            //   mp:    is a reference to the whole multi_pass instance
678            //   rhs:   is the multi_pass reference this instance is compared
679            //          to
680            //
681            // This function is expected to return true if the current instance
682            // is equal to the right hand side multi_pass instance
683            template <typename MultiPass>
684            static bool equal_to(MultiPass const& mp, MultiPass const& rhs);
685
686            // less_than is called whenever multi_pass::operator<() is invoked
687            //
688            //   mp:    is a reference to the whole multi_pass instance
689            //   rhs:   is the multi_pass reference this instance is compared
690            //          to
691            //
692            // This function is expected to return true if the current instance
693            // is less than the right hand side multi_pass instance
694            template <typename MultiPass>
695            static bool less_than(MultiPass const& mp, MultiPass const& rhs);
696        };
697
698        // Value is the same type as typename MultiPass::value_type
699        template <typename Value>
700        struct shared {};
701    };
702
703It is possible to derive the struct `unique` from the type
704`boost::spirit::detail::default_storage_policy`. This type implements a
705minimal sufficient interface for some of the required functions, simplifying
706the task of writing a new storage policy.
707
708This class may implement a function `destroy()` being called during destruction
709of the last copy of a `multi_pass`. This function should be used to free any of
710the shared data items the policy might have allocated during construction of
711its `shared` part. Because of the way `multi_pass` is implemented any allocated
712data members in `shared` should _not_ be deep copied in a copy constructor of
713`shared`.
714
715Generally, a `StoragePolicy` is the trickiest policy to implement. You should
716study and understand the existing `StoragePolicy` classes before you try and
717write your own.
718
719
720[endsect]
721
722
723
724