doc/book/chapter_vsl.texi

@chapsummary
This section describes how to save and restore objects using a binary stream.
It details how to add the appropriate functions to each class to make use
of this facility.
@endchapsummary

All objects in VXL should be able to save themselves to a stream (eg a file)
and restore (load) themselves from a stream (file). The main functions provided
for this purpose are @code{vsl_b_write(os,object);} and @code{vsl_b_read(is,object&);}.

The binary IO for the core libraries (vbl, vil, vgl and vnl) is implemented in `clip-on'
libraries which live in the @code{io} subdirectories of each library (thus the
declaration of the function @code{vsl_b_write(vsl_b_ostream&,vnl_vector const&);}
lives in the file @file{vnl/io/vnl_io_vector.h}.

However, it is recommended that I/O for other libraries be provided by writing
@code{b_write(os);} and @code{b_read(is);} functions in each class.  See the
`Design Notes' section below.

@section Supported Platforms

The binary I/O code is known to work across the following hardware/OS/compiler
combinations, but probably also works on most other platform/compiler combinations:

@enumerate
@item  THIS CODE IS OUT OF DATE!
@end enumerate

Thus
binary files produced by any of the above should be readable by any other of the above.
There is of course a minor exception: large numbers (like integers larger than 4294967295)
saved on a 64-bit platform cannot be read on a 32-bit platform.

@subsection caveats

The code has been designed to work on as many platforms as possible. However if your
platform uses any of the following, then it will probably not work (as presently coded.)
@enumerate
@item  A middle endian word encoding scheme.
@item  Chars of length other than 8 bits.
@item  Non-IEEE format floats and doubles.
@end enumerate

@section Using Binary I/O

To save an object to a file, simply do the following:
@example
  vxl_myclass my_object;

  // Fill my_object

  vsl_b_ofstream bfs("my_object.bvl");
  if (!bfs)
  @{
    vcl_cerr<<"Failed to open my_object.bvl for output."<<vcl_endl;
  @}
  else
  @{
    vsl_b_write(bfs,my_object);
    bfs.close();
  @}
@end example

To load/restore the object from a file:

@example
  vxl_myclass my_new_object;

  vsl_b_ifstream bfs("my_object.bvl");
  if (!bfs)
  @{
    vcl_cerr<<"Failed to open my_object.bvl for input."<<vcl_endl;
  @}
  else
  @{
    vsl_b_read(bfs,my_object);
    bfs.close();
  @}
@end example

It is recommended that the default extension name for your binary files is @code{.bvl}.
This extension does not appear to be used by any other program. In many cases however, you
will want to pick a new extension to indicate the contents of a file. For example, we store
active shape model objects with ending @code{.asm}.

The classes @code{vsl_b_ifstream} and @code{vsl_b_ofstream} are simple wrappers around
real @code{vcl_ifstream} and @code{vcl_ofstream} objects. These wrappers ensure that
you open a file with CR/LF conversion turned off, and they should also allow lots of
common misuses to be caught at compile time.

The functions @code{vsl_b_write(os,X)} and @code{vsl_b_read(is,X)} are defined for all
reasonable cases, including all inbuilt types, most classes in vcl and the classes in
the core vxl libraries.

When you write a new class, you should add the appropriate functions to allow easy
use of binary I/O (see below).

Or for simplicity we provide the utility functions which would allow you to write:

@example
  #include <vsl/vsl_quick_file.h>
  vxl_myclass my_object,my_new_object;

  vsl_quick_file_save("my_object.bvl",my_object);
  vsl_quick_file_load("my_object.bvl",my_new_object);
@end example


@subsection Saving multiple objects

One can use exactly the same approach to save a set of objects

@example
  vxl_myclass my_object;
  vxl_my_other_class my_other_object;

  // Fill objects
  // ...

  vsl_b_ofstream bfs("my_object.bvl");
  if (!bfs)
  @{
    vcl_cerr<<"Failed to open my_object.bvl for output."<<vcl_endl;
  @}
  else
  @{
    vsl_b_write(bfs,my_object);
    vsl_b_write(bfs,my_other_object);
    bfs.close();
  @}
@end example

(and similarly for loading them).

A standard rule for ensuring trouble free I/O is
@quotation
Always write the input and output code in tandem - the output should
precisely mirror the input.
@end quotation


@subsection Binary I/O by baseclass pointer

When using polymorphism, there are frequently times when one needs to
save and restore an object just using a base class pointer to it.
@code{vsl} provides facilities to do this.

Assuming class @code{my_derived} is derived from class @code{my_base}, the following
will work.

To save an object by baseclass:

@example
  my_derived d;

  my_base *b = &d;

  vsl_b_ofstream bfs("data.bvl");
  vsl_b_write(bfs,b);
  ...
@end example

To restore an object:
@example
  // Make application aware of possible classes that it might see in the file

  vsl_add_to_binary_loader(my_derived());
  vsl_add_to_binary_loader(my_derived2());
  ...

  my_base *b = 0;

  vsl_b_ifstream bfs("data.bvl");
  vsl_b_read(bfs,b);
  // b now points to the correct class which has been created
  // on the heap and filled with the data from bfs
  ...
@end example

Note that the read function will only work if the application has been made
aware of each of the possible derived classes that it might come across in the
file.  This is done using calls to @code{vsl_add_to_binary_loader(my_derived())}
(see appendix for details).

To reduce the pain of doing this, many libraries have a function that adds all
the relevant derived classes (eg @code{xxxx_add_all_binary_loaders()} where
@code{xxxx} is the library name).

@subsection Which files do I need to include/link?

In general the vsl_b_read and vsl_b_write functions use Koenig Lookup - that is the location
of their declaration depends on their parameters.

The @code{vsl_b_stream} objects and @code{vsl_b_write} and @code{vsl_b_read} functions
for fundamental data types are declared in @code{<vsl/vsl_binary_io.h>}. If you want to
load or save a @code{vcl_vector}, the appropriate @code{vsl_b_write} and @code{vsl_b_read}
functions will be in @code{<vsl/vsl_vector_io.h>}. Likewise for most of the other vcl classes.
The @code{vsl} library contains the implementation of all of this.

When reading/writing by baseclass pointer, you need to include @code{vsl/vsl_binary_loader.h}.

If you want to load or save a @code{vgl_point_2d}, you will need to include
@code{<vgl/io/vgl_io_point_2d.h>} and similarly for all other Level-1 VXL libraries. You
will need to include the vgl_io library.
For Level-2 libraries, the situation varies. If binary io has been defined at all for a
level-2 library, it might be included in the library itself, e.g. the io functions for
vpdfl_gaussian are declared in the same file as the Gaussian, @code{vpdfl/vpdfl_gaussian.h>}.
Alternatively, it might be in a clip-on library in the same form as the Level-1 libraries
above.

@subsection How to save templated objects

The situation for templated objects is the same as above, except that you need to ensure that
the appropriate (templated) @code{vsl_b_read} and @code{vsl_b_write} functions are explicitly instantiated.
This instantiation is achieved by placing a file in the relevant "Templates" folder.

An example template file, is shown below. It enables saving of a 2d array of "@code{hjk_model}"s
(a completely made up plain class).

@example
  // This is my_module/hjk/Templates/vbl_array_2d_io+hjk_model~-.cxx
  #include <vbl/io/vbl_io_array_2d.hxx>
  #include <hjk/hjk_model.h>
  VBL_IO_ARRAY_2D_INSTANTIATE(hjk_model);
@end example

The @code{vbl_io_array_2d.txx} file contains the @code{VBL_IO_ARRAY_2D_INSTANTIATE} macro and the
@code{hjk_model.h} file contains the io header declarations for a plain class.

Another example template file, allowing the saving of a vector of @code{vgl_point_2d} objects,
is shown below.

@example
  // file = my_module/hjk/Templates/vsl_vector_io+vgl_point_2d~-.cxx
  #include <vsl/vsl_vector_io.hxx>
  #include <vgl/io/vgl_io_point_2d.h>
  VSL_VECTOR_IO_INSTANTIATE(vgl_point_2d<double>);
@end example

The @code{vsl_vector_io.txx} file contains the @code{VSL_VECTOR_IO_INSTANTIATE} macro and the
@code{vgl_io_point_2d.h} file contains the io header declarations for @code{vgl_point_2d<double>}.

You should now be able to load and save templated objects with lines such as:-

@example
  vcl_vector<hjk_model> hjk_model_vec;
  vsl_b_ofstream bfs("hjk_model_vec.bvl");
  if (!bfs)
  @{
    vcl_cerr<<"Failed to open hjk_model_vec.bvl for output."<<vcl_endl;
  @}
  else
  @{
    vsl_b_write(bfs,hjk_model_vec);
    bfs.close();
  @}
@end example

NB, the template instantiation files should be placed in your own libraries (ie here "hjk") to
avoid creating unnecessary and unused versions of a given templated function.


@section Tidy Printing with @code{vsl_indent}

The utility functions and class in @code{vsl_indent} give a way of putting
indentation into output streams to give more legible printed output.

If each class implements it's printing (print(os) or print_summary(os))
in such a way that at the beginning of each new line one inserts an
indentation using
@example
  os<<vsl_indent()<<"Rest of stuff.."<<vcl_endl;
@end example
and increases and decreases the current indentation for the stream with
@code{vsl_indent_inc(os)} and @code{vsl_indent_dec(os)},
then one can easily generate readable output for complex nested sets
of classes.

It's use is best described by example:

@example
  vcl_cout<<vsl_indent()<<"No Indent"<<vcl_endl;
  vsl_indent_inc(vcl_cout);
  vcl_cout<<vsl_indent()<<"1 Indent"<<vcl_endl;
  vsl_indent_inc(vcl_cout);
  vcl_cout<<vsl_indent()<<"2 Indent"<<vcl_endl;
  vsl_indent_dec(vcl_cout);
  vcl_cout<<vsl_indent()<<"1 Indent"<<vcl_endl;
  vsl_indent_dec(vcl_cout);
  vcl_cout<<vsl_indent()<<"No Indent"<<vcl_endl;
@end example

This produces output of the form
@example
No Indent
  1 Indent
    2 Indent
  1 Indent
No Indent
@end example

Example of use in class output:
@example
  class Fred
  @{
   public:
    void print(vcl_ostream& os) const @{ os<<vsl_indent(os)<<"Fred's data"; @}
  @};

  vcl_ostream& operator<<(vcl_ostream& os, Fred const& fred)
  @{
    os<<"Fred: "<<vcl_endl;
    vsl_indent_inc(os);
    fred.print(os);
    vsl_indent_dec(os);
    return os;
  @}

  class Jim
  @{
   private:
    Fred fred_;
   public:
    void print(vcl_ostream& os) const
    @{
       os<<vsl_indent()<<fred_<<vcl_endl
           vsl_indent()<<"Jim's other data";
    @}
  @};

  vcl_ostream& operator<<(vcl_ostream& os, Jim const& jim)
  @{
    os<<"Jim: "<<vcl_endl;
    vsl_indent_inc(os);
    jim.print(os);
    vsl_indent_dec(os);
    return os;
  @}

  main()
  @{
    Jim jim;
    vcl_cout<<jim<<vcl_endl;
  @}
@end example

This produces output:
@example
 Jim:
   Fred's data
   Jim's other data
@end example

If Jim were then included as a member of another class, Harry, one could get
output of the form
@example
  Harry:
   Harry's basic data
   jim1:
     Fred's data
     Jim's other data
   jim2:
     Fred's data
     Jim's other data
@end example

and so forth.  The author humbly suggests that this makes the summaries
quite readable.

@section Error Detection

IO is often prone to errors beyond the control of the programmer. In
particular, files can be come corrupted, given to programs that can't read
a new format, read on platforms that do not support large enough numbers.

vsl attempts to detect as many error conditions as possible.
It prints an error message to @code{vcl_cerr} and sets the fail bit
on the input stream. Any objects that were being loaded when the error
occurred should be consistent at least as far as being able to delete the
object safely.

During the opening of a binary input stream, vsl also checks
for a schema version number, and magic number that confirm that
the stream was written by vsl.

It is easy to detect the error condition as the example shows
@example
  vsl_b_ifstream bfs_in(path);
  if (!bfs_in)
  @{
    vcl_cout << "Could not open " << path
             << " for reading as binary IO" << vcl_endl;
    return;
  @}
  vsl_b_read(bfs_in, my_obj);
  if (!bfs_in)
  @{
    vcl_cout << "Unable to read my_obj" << vcl_endl;
    return;
  @}
  bfs_in.close();
@end example