xref: /dports/security/bsdsfv/bsdsfv/MANUAL

$Id: MANUAL,v 1.1.1.1 2002/05/21 13:41:21 webbie Exp $

                  BSD SFV Builder for UNiX / LiNUX
             [ formerly known and hated as Hoopy SFV :) ]




========================================================================

                             DOCUMENTATiON

========================================================================


0. Background Information
-------------------------

When I started working on some ISO upload tools, the topic of site advertise-
ment banners in *.sfv came into my mind. Some sites were putting their logo
as comment into each SFV file uploaded to them, so if you had a bad day and
a release passed other sites before it came to you, you got a 5 kb .SFV file
with several site banners in it... and that sucked.
Since I already made some stuff to remove unwanted .nfo's from ZIP uploads, I
decided to filter all comments out of uploaded .SFV files, so only the real
filenames and their checksum stayed in them.
That worked fine for my sites, files got CRC-checked by "sfv_check" which
comes with the ftp demon I used, glFTPd, without any problems. A few days
later a friend who had downloaded some stuff complained he couldn't check
it using WinSFV and SFV*Nix just gave an error message

          "./sfv: test.sfv: Unable to identify type of input file" .

I never used neither WinSFV nor SFV*NIX before since I always relied on the
site to do the CRC check, so there was no need to check the stuff at home as
well. When I compared a SFV file working with the one not working, I figured
out that there was a special line

                     "; Generated by WIN-SFV32 ..."

at the very top of the working SFV file. I should have run some deeper analysis
of that, but in the mood of that time I just decided to put a "; Generated by
HoopySFV" line to the top of each uploaded SFV file after stripping all other
comments out, hoping it would satisfy any program looking for a "; Generated by"
line in a SFV file.

Guess I was wrong with that, since when my tools were used on some more sites,
more and more people came complaining about the lacking possibility to check
"HoopySFV" - created SFV files with either Win-SFV or Sfv*Nix. This time I had
a closer look at the source of SFV*NIX which is available to public so you
can compile it on your Unix system yourself. Having a look at "parse.c" showed
that SFV*NIX (and I guess WIN-SFV as well) is not only looking for a line
starting with "; Generated by", but also for the name of either winsfv or
sfvnix. That meant SFV*NIX/WIN-SFV would refuse to process any SFV file not
created by either of them (there is support for Validate and FileCRC as well,
but who uses them?). To put it in other words, with SFV*NIX/WIN-SFV you can't
process any SFV file which doesn't contain the name of either of them in it.

Since I was not very in fond of that because SFV*NIX/WIN-SFV might be the
current standard software, but there could be others as well, a friend of mine
arranged a meeting with the author of sfvnix/winsfv, MrAlc, and we discussed
that matter. I pointed out that it's rather useless to try to identify a .sfv
file by searching for a "Generated by WIN-SFV" in it because the filename
itself should be enough, and he agreed to that and "promised" to release a fix
for that "soon". It's now more than two months later and I know exactly two
persons who got a quick patch for SFV*NIX. There are still people coming to
me and complain about checking "HoopySFV" files with WIN-SFV, so I guess the
promised fix is still not released to public.

Please note that there was no "HoopySFV" existing before this release; the
line "; Generated by HoopySFV" just got appended to uploaded SFV files on
some sites, just like banners get appended to them. And it was done because
I wanted to satisfay any sfv-checker which needed a "generated by" - line and
not because I needed to see my nick in every damn sfv file. For that reason
I renamed the whole stuff to "public domain SFV builder", so you can do with
it whatever you want.

Several people have asked me for a patch to SFV*NIX since I knew there was
just a little modification to that parse.c file necessary, but since I
neither do have the source of WIN-SFV nor like to tamper with other's source
codes, I decided to do a little SFV program myself. It doesn't have that
many features SFV*NIX has since I wondered who'd need them all, so what you
can do with this one is creating SFV files, check single files against their
CRC listed in a SFV, check all files listed in a SFV and count the number
of existing / missing files. And if you're pissed off one day and don't want
to use pdSFV anymore, you don't need to worry about compatibility with other
programs. pdSFV can read them all and is pretty flexible in the files it
creates.

I've included the full source for pdSFV/Unix and I tried to keep its routines
as simple as possible and easy to read. I want to encourage any skilled
and interested programmer out there to do his own SFV utility, since there
is nothing worse than depending on a single or just a few persons concerning
every-day used software. If you implement nice new features, I'd appreciate
it if you'd send your stuff to me. I've only made this one due to the many
requests and usually focus on developing other stuff, so if you're willing
to develop a better allround SFV tool, any cooperation is welcome.


1. SFV file format
------------------

A SFV file is used to store the CRC-32 checksum of files. This way you're
able to send whichever files to a friend and he can check out whether the
files he received are identical to the ones you sent by comparing the CRC
of the file he got with the one listed in the SFV file.

Basically a SFV file contains a list of filenames and their CRC codes.
It is a plain ASCII file with one filename per line with the CRC code
appended to each filename, seperated by a single blank (ascii code 20h).

Example:

File01.rar DB12255D
File05.rar 7E4CA2BB


Comments can be added to SFV files. They must be on a line of their own
and start with "; ", for example:

; This is a comment line

It is common that the very first line of each SFV file is the "generated
by" comment line:

; Generated by MySFV

or

; Generated by Lamer01 during holiday in Spain on 07/11/99 at 17:08:54


The point about it is that besides the list of files and their checksums
nothing is really needed in a SFV file. Some FTP sites add their banners
as comments. Some SFV tools add the file size, date and time as comments
to it. If you're writing an utility for SFV stuff yourself, you should
not rely on anything being in there besides the filenames and the CRCs.
If you do and catch a SFV which doesn't meet your requirements, work on
your stuff and don't blame any "fucked up format".



2. Using pdSFV
--------------

The stuff you got here is raw meat. It's one of my quick&dirty productions
and surely not lamer-proof. If handled improperly, it'll prolly segfault
and refuse to do what you want it to do. It's working fine for me btw :-)
You've got the source, so make out of it whatever you want. Improve it and
let the ppl have it.

Before you can use it, you have to compile the source. To do so, issue the
following command after unpacking pdsfv.c to your directory:

gcc -o pdsfv pdsfv.c

(if you don't have gcc installed, try "cc -o pdsfv pdsfv.c"). I've
tested it under several Linux installations and the following Unixes:

HP-UX B.10.20 A 9000/720
SunOS 5.7 Generic_106541-05 sun4u sparc SUNW,Ultra-60

It was successfully compiled and used by Onyx^CNCiSO under the following OSs:

IRIX 6.5 IP32
SunOS 5.6 (Sparc ULTRA2)
OSF1 (Alpha V4.0)

If you have problems compiling or using it under your Unix, please let me know
and have the compiler's error and warning messages handy.


Try a "./pdsfv" to check out whether it worked that far. You should get
something similar to:

===
Usage: ./pdsfv [mode | option] <sfv-filename> <filename(s)>

Possible modes: -c : create SFV file
                -t : test single file(s) (multiple filenames possible)
                -T : test whole SFV file
                -m : count missing files

Options:        -l <filename> : use <filename> as banner file (create mode)
                -w : Win-SFV emulation mode                   (create mode)
                -a : add all files to .sfv (even .nfo etc.)   (create mode)
===


Now let's go through the different operational modes...



CREATING SFV FILES
------------------

We need to use "pdsfv -c mysfvname.sfv filenames.*" basically to do so. Let's
say we have some files myrls.rar, myrls.r00, myrls.r01, myrls.r02 which we
want to get listed in myrls.sfv ...

We then could use

pdsfv -c myrls.sfv myrls.rar myrls.r00 myrls.r01 myrls.r02

or shorter:

pdsfv -c myrls.sfv myrls.r*

We will get an output like this:

---
Creating new checksum file: myrls.sfv

Adding file: myrls.r00 ... CRC = 0x4C27B20A
Adding file: myrls.r01 ... CRC = 0x3BB960FA
Adding file: myrls.r02 ... CRC = 0xA01C2C95
Adding file: myrls.rar ... CRC = 0xD782FE65

SFV file successfully created (4 files processed) ...
---

Let's have a look at the SFV file created:

---
; Generated by pdSFV/Unix on Thu Oct 21 19:26:48 1999
;
myrls.r00 4C27B20A
myrls.r01 3BB960FA
myrls.r02 A01C2C95
myrls.rar D782FE65
---

That's it :) Now for some of the options:


Adding logos/banners
--------------------

If you have some nice ASCII banner, you can add it on top of the created
SFV file by using the -l command line option. Let's say your bannerfile
is /home/lamer/mygroup.nfo, then you would use:

pdsfv -c myrls.sfv -l /home/lamer/mygroup.nfo myrls.r*

to create the SFV file. You will get this confirmation:

---
Creating new checksum file: myrls.sfv
Adding banner file        : /home/lamer/mygroup.nfo
.
.
--


Adding "all" files
------------------

If called without the -a command line switch, pdSFV only adds *.r?? and *.0??
files to the SFV. If you do a

pdsfv -c myrls.sfv -a myrls.r* myrls.nfo

even myrls.nfo's CRC will be stored in the SFV file. Please note that this is
generally a bad idea to do for various reasons. That's why it's not default and
you have to use -a if you want to do so.

Also note that you can include myrls.sfv in itself if using -a . This obviously
is pretty lame, so you might want to try to avoid it :-)


WIN-SFV compatibility mode
--------------------------

Since still many people are using Winsfv/Sfvnix and as long as there is no
new version of them out which support SFV files created by other tools, you
can switch to compatibility mode by using -w ...

pdsfv -c myrls.sfv -w myrls.r*

will then generate something like:

; Generated by WIN-SFV32 v1.5 on Thu Oct 21 19:38:23 1999
instead of
; Generated by pdSFV/Unix on Thu Oct 21 19:38:23 1999

You'll get the confirmation:

---
Creating new checksum file: myrls.sfv
Using compatibility mode  : Win-SFV
---



TESTING SINGLE FILES
--------------------

If you want to verify that one or more single files are good, you can use mode
-t ...

Examples:

---
pdsfv -t myrls.sfv myrls.rar

Testing myrls.rar ... local = 0xD782FE65, listed = 0xD782FE65 - OK

1 file(s) tested - 1 OK - 0 bad...
---
pdsfv -t myrls.sfv myrls.rar myrls.r00

Testing myrls.rar ... local = 0xD782FE65, listed = 0xD782FE65 - OK
Testing myrls.r00 ... local = 0x8208D936, listed = 0x4C27B20A - BAD

2 file(s) tested - 1 OK - 1 bad...
---

You will get errorlevel 0 if all files were good, and errorlevel 1 if
one or more of the tested files were bad. You can use this in your
upload check / zipscript stuff.

For glFTPd users: if you modify your zipscript to use pdSFV instead of
the sfv_check that comes with glFTPd, note that you need command line
parameters with pdSFV and not just a single filename. Replace

cd $2
/bin/sfv_check $1

with

cd $2
/bin/pdsfv -t *.[sS][fF][vV] $1

to get it working!



TESTING A WHOLE RELEASE
-----------------------

Use -T to test all the files listed in the sfv file!

---
pdsfv -T myrls.sfv

Processing complete check of myrls.sfv ...
Testing myrls.r00 ... listed = 0x4C27B20A ... local = 0x8208D936 ... BAD
Testing myrls.r01 ... listed = 0x3BB960FA ... local = 0x3BB960FA ... OK
Testing myrls.r02 ... listed = 0xA01C2C95 ... local = 0x00000000 ... MISSING
Testing myrls.rar ... listed = 0xD782FE65 ... local = 0xD782FE65 ... OK

4 file(s) tested - 2 OK - 1 bad - 1 missing ...
---

Errorlevels:    0 - all files there and OK
                1 - one or more files bad
                2 - one or more files missing



COUNTING MISSING FILES
----------------------

---
pdsfv -m myrls.sfv

Performing completion check...
[total files listed] [local files] [missing files]
4
3
1
---

You can use this one in shellscripts to fetch the number of total
files in this release, the number of how many files you've got
local yet and how many are missing.

To get the number of files existing locally (3 in our example), you
could use some shit like:

NUMLOCAL=`pdsfv -m myrls.sfv | tail -n 2 | head -n 1`
echo "$NUMLOCAL files of this release have arrived yet!"

If you're a funny shellscripter you surely can use this somehow. Just
note that the files are not tested when using -m ! This means if there
are some bad files lying around or the files are being uploaded at the
moment and not finished yet, they will count as "there" and not as
"missing".