This is an alpha release of TLB (List Batcher).  It has been lightly
tested; use with care.


What is TLB?

TLB is a program which accepts a message and sends it to a list of
addresses, using multiple SMTP transactions to do so.  The idea is to allow
multiple deliveries in parallel.  It is in some ways similar to the program
bulk_mailer by Keith Moore and was indeed inspired by it, but were
bulk_mailer simply calls Sendmail multiple times, TLB can open SMTP
connections to any host (or any program) which allows it and have that host
manage the final delivery of the message.

TLB allows flexibility in which and how many addresses are sent in each
SMTP transaction.  Addresses are split into distinct lists according to
regular expressions.  These lists are then sorted so that like domains
cluster together, and finally the lists are split into batches of
configurable sizes and properties.  These batches are then distributed
evenly among lists of hosts which will manage final delivery.  This system
allows for remote exploders for overseas addresses and other useful
configurations.

TLB can also pipe the message to a list of local programs.  The Majordomo
list manager (and perhaps others; I've no familiarity with them) generally
performs functions like archiving and digestification through programs that
are included along with the list of addresses in the outgoing alias.  Since
TLB replaces the list of addresses, it makes sense to also allow it to
replace the other programs.  This allows the complete elimination of the
normal outgoing aliases associated with a list (and thus the complete
elimination of the security headaches normally associated with the presence
of outgoing aliases).  This has not been tested with digested lists at all,
though it should work.  It does work with moderated lists.


Requirements:

TLB requires Perl version 5.003.  Version 5.002 should work but has not
been tested.  Versions less than 5.002 will definitely not work without
some source hackery, as TLB makes use of subroutine prototyping.  Perl4?
Forget it.  TLB makes extensive use of complex data structures and
references which just aren't doable in ancient versions of Perl.

The standard packages Getopt, POSIX, Sys::Syslog, Socket and IPC::Open2 are
used.  These come with Perl; if you don't have them then you should check
to see that Perl was installed properly at your site.  Note also that
Sys::Syslog requires that your system headers be run through the h2ph
program; this is normally done in any Perl installation but it is not done
automatically and therefore is occasionally forgotten.

In addition, routines from the packages IO, TimeDate, MailTools and libnet
are used. These packages are available from CPAN, the Comprehensive Perl
Archive Network.  Look at <URL:http://www.perl.com/perl/> for more
information on accessing CPAN.  Possibly out of date copies are kept in
<URL:ftp://ftp.hpc.uh.edu/pub/tlb/>.  In particular, the IO package must be
at least version 1.09 due to bugs in the socket code of earlier versions.

The Time::Hires package is also optionally supported; with it, TLB can
generate more accurate timing data.


Bugs:

TLB probably has bugs.  I have tested most of the features through what I
believe to be normal operating conditions and it performs without problems.
I have also tested it under some error conditions (mainly down hosts and
hosts that refuse connection) without problems.  TLB will currently act
unpredictably if a hosts accepts a connection but will not accept mail
(either through an error greeting or through some error during the SMTP
exchange).  It will most likely drop the batch and continue without marking
the host down or enabling backup hosts.  This will change.

TLB runs cleanly with all warnings enabled and with all strict checks.  It
has not been tested with taint checking, though it is not really necessary
since it is assumed that tlb will run with no special privileges.

As with any program not written in machine language, TLB is sensitive to
problems in the language it is written in.  Perl is a complex system and
has bugs of its own, some of which interfere with TLB.  There is a problem
on some systems where perl randomly crashes when external programs are
called repeatedly.  Please contact me if you experience any problems of
this nature.

Report all problems to me, Jason Tibbitts, at tibbs@hpc.uh.edu.  There is
not currently a mailing list for TLB, but one will be started if there is
enough demand.

This shows the information that is logged at each log_level.

warning
 1 Abort
 2 Failsafe mode
notice
 3 Marking host down
   Socket open error
   Too long waiting for child exit
   SMTP response timeout
 4 Non-fatal SMTP response
   Fatal SMTP Response
 5 Single rejected address
info
 6 Compilation times
   Message reading time
   Address reading time
   Batching time
   External spawning time
   Total delivery time
 7 Dest delivery time
 8 Starting
   Batch delivery time
 9 Connection opening time
10 SMTP command turnaround time
11 SMTP Command
12 SMTP Response

The mungelist program can perform two operations on a list of addresses:

1. Strip comments each address
2. Sort the list in reverse-domain order

For very large lists the time savings caused by eliminating the sorting
process may be significant, and a list that is not completely sorted does
not overly disrupt the batching process.  Thus you can sort your list at
various intervals and tell TLB to skip the sorting process.  You should
check the logs to see just how much time TLB is spending sorting your
addresses.

It may be the case that stripping the addresses takes a significant anount
of time.  TLB really prefers to deal with only stripped addresses, so if
you have unstripped addresses in a list but you tell TLB that it has been
stripped, the results will be unsatisfactory.

Mungelist takes two options (you may specify one or both):

 -o  sOrt the list of addresses
 -t  sTrrip comments from the addresses

It takes one argument, the name of the address file to process.  The
processed list will appear on standard output.