1$Id: MANUAL,v 1.1.1.1 2002/05/21 13:41:21 webbie Exp $ 2 3 BSD SFV Builder for UNiX / LiNUX 4 [ formerly known and hated as Hoopy SFV :) ] 5 6 7 8 9======================================================================== 10 11 DOCUMENTATiON 12 13======================================================================== 14 15 160. Background Information 17------------------------- 18 19When I started working on some ISO upload tools, the topic of site advertise- 20ment banners in *.sfv came into my mind. Some sites were putting their logo 21as comment into each SFV file uploaded to them, so if you had a bad day and 22a release passed other sites before it came to you, you got a 5 kb .SFV file 23with several site banners in it... and that sucked. 24Since I already made some stuff to remove unwanted .nfo's from ZIP uploads, I 25decided to filter all comments out of uploaded .SFV files, so only the real 26filenames and their checksum stayed in them. 27That worked fine for my sites, files got CRC-checked by "sfv_check" which 28comes with the ftp demon I used, glFTPd, without any problems. A few days 29later a friend who had downloaded some stuff complained he couldn't check 30it using WinSFV and SFV*Nix just gave an error message 31 32 "./sfv: test.sfv: Unable to identify type of input file" . 33 34I never used neither WinSFV nor SFV*NIX before since I always relied on the 35site to do the CRC check, so there was no need to check the stuff at home as 36well. When I compared a SFV file working with the one not working, I figured 37out that there was a special line 38 39 "; Generated by WIN-SFV32 ..." 40 41at the very top of the working SFV file. I should have run some deeper analysis 42of that, but in the mood of that time I just decided to put a "; Generated by 43HoopySFV" line to the top of each uploaded SFV file after stripping all other 44comments out, hoping it would satisfy any program looking for a "; Generated by" 45line in a SFV file. 46 47Guess I was wrong with that, since when my tools were used on some more sites, 48more and more people came complaining about the lacking possibility to check 49"HoopySFV" - created SFV files with either Win-SFV or Sfv*Nix. This time I had 50a closer look at the source of SFV*NIX which is available to public so you 51can compile it on your Unix system yourself. Having a look at "parse.c" showed 52that SFV*NIX (and I guess WIN-SFV as well) is not only looking for a line 53starting with "; Generated by", but also for the name of either winsfv or 54sfvnix. That meant SFV*NIX/WIN-SFV would refuse to process any SFV file not 55created by either of them (there is support for Validate and FileCRC as well, 56but who uses them?). To put it in other words, with SFV*NIX/WIN-SFV you can't 57process any SFV file which doesn't contain the name of either of them in it. 58 59Since I was not very in fond of that because SFV*NIX/WIN-SFV might be the 60current standard software, but there could be others as well, a friend of mine 61arranged a meeting with the author of sfvnix/winsfv, MrAlc, and we discussed 62that matter. I pointed out that it's rather useless to try to identify a .sfv 63file by searching for a "Generated by WIN-SFV" in it because the filename 64itself should be enough, and he agreed to that and "promised" to release a fix 65for that "soon". It's now more than two months later and I know exactly two 66persons who got a quick patch for SFV*NIX. There are still people coming to 67me and complain about checking "HoopySFV" files with WIN-SFV, so I guess the 68promised fix is still not released to public. 69 70Please note that there was no "HoopySFV" existing before this release; the 71line "; Generated by HoopySFV" just got appended to uploaded SFV files on 72some sites, just like banners get appended to them. And it was done because 73I wanted to satisfay any sfv-checker which needed a "generated by" - line and 74not because I needed to see my nick in every damn sfv file. For that reason 75I renamed the whole stuff to "public domain SFV builder", so you can do with 76it whatever you want. 77 78Several people have asked me for a patch to SFV*NIX since I knew there was 79just a little modification to that parse.c file necessary, but since I 80neither do have the source of WIN-SFV nor like to tamper with other's source 81codes, I decided to do a little SFV program myself. It doesn't have that 82many features SFV*NIX has since I wondered who'd need them all, so what you 83can do with this one is creating SFV files, check single files against their 84CRC listed in a SFV, check all files listed in a SFV and count the number 85of existing / missing files. And if you're pissed off one day and don't want 86to use pdSFV anymore, you don't need to worry about compatibility with other 87programs. pdSFV can read them all and is pretty flexible in the files it 88creates. 89 90I've included the full source for pdSFV/Unix and I tried to keep its routines 91as simple as possible and easy to read. I want to encourage any skilled 92and interested programmer out there to do his own SFV utility, since there 93is nothing worse than depending on a single or just a few persons concerning 94every-day used software. If you implement nice new features, I'd appreciate 95it if you'd send your stuff to me. I've only made this one due to the many 96requests and usually focus on developing other stuff, so if you're willing 97to develop a better allround SFV tool, any cooperation is welcome. 98 99 1001. SFV file format 101------------------ 102 103A SFV file is used to store the CRC-32 checksum of files. This way you're 104able to send whichever files to a friend and he can check out whether the 105files he received are identical to the ones you sent by comparing the CRC 106of the file he got with the one listed in the SFV file. 107 108Basically a SFV file contains a list of filenames and their CRC codes. 109It is a plain ASCII file with one filename per line with the CRC code 110appended to each filename, seperated by a single blank (ascii code 20h). 111 112Example: 113 114File01.rar DB12255D 115File05.rar 7E4CA2BB 116 117 118Comments can be added to SFV files. They must be on a line of their own 119and start with "; ", for example: 120 121; This is a comment line 122 123It is common that the very first line of each SFV file is the "generated 124by" comment line: 125 126; Generated by MySFV 127 128or 129 130; Generated by Lamer01 during holiday in Spain on 07/11/99 at 17:08:54 131 132 133The point about it is that besides the list of files and their checksums 134nothing is really needed in a SFV file. Some FTP sites add their banners 135as comments. Some SFV tools add the file size, date and time as comments 136to it. If you're writing an utility for SFV stuff yourself, you should 137not rely on anything being in there besides the filenames and the CRCs. 138If you do and catch a SFV which doesn't meet your requirements, work on 139your stuff and don't blame any "fucked up format". 140 141 142 1432. Using pdSFV 144-------------- 145 146The stuff you got here is raw meat. It's one of my quick&dirty productions 147and surely not lamer-proof. If handled improperly, it'll prolly segfault 148and refuse to do what you want it to do. It's working fine for me btw :-) 149You've got the source, so make out of it whatever you want. Improve it and 150let the ppl have it. 151 152Before you can use it, you have to compile the source. To do so, issue the 153following command after unpacking pdsfv.c to your directory: 154 155gcc -o pdsfv pdsfv.c 156 157(if you don't have gcc installed, try "cc -o pdsfv pdsfv.c"). I've 158tested it under several Linux installations and the following Unixes: 159 160HP-UX B.10.20 A 9000/720 161SunOS 5.7 Generic_106541-05 sun4u sparc SUNW,Ultra-60 162 163It was successfully compiled and used by Onyx^CNCiSO under the following OSs: 164 165IRIX 6.5 IP32 166SunOS 5.6 (Sparc ULTRA2) 167OSF1 (Alpha V4.0) 168 169If you have problems compiling or using it under your Unix, please let me know 170and have the compiler's error and warning messages handy. 171 172 173Try a "./pdsfv" to check out whether it worked that far. You should get 174something similar to: 175 176=== 177Usage: ./pdsfv [mode | option] <sfv-filename> <filename(s)> 178 179Possible modes: -c : create SFV file 180 -t : test single file(s) (multiple filenames possible) 181 -T : test whole SFV file 182 -m : count missing files 183 184Options: -l <filename> : use <filename> as banner file (create mode) 185 -w : Win-SFV emulation mode (create mode) 186 -a : add all files to .sfv (even .nfo etc.) (create mode) 187=== 188 189 190Now let's go through the different operational modes... 191 192 193 194CREATING SFV FILES 195------------------ 196 197We need to use "pdsfv -c mysfvname.sfv filenames.*" basically to do so. Let's 198say we have some files myrls.rar, myrls.r00, myrls.r01, myrls.r02 which we 199want to get listed in myrls.sfv ... 200 201We then could use 202 203pdsfv -c myrls.sfv myrls.rar myrls.r00 myrls.r01 myrls.r02 204 205or shorter: 206 207pdsfv -c myrls.sfv myrls.r* 208 209We will get an output like this: 210 211--- 212Creating new checksum file: myrls.sfv 213 214Adding file: myrls.r00 ... CRC = 0x4C27B20A 215Adding file: myrls.r01 ... CRC = 0x3BB960FA 216Adding file: myrls.r02 ... CRC = 0xA01C2C95 217Adding file: myrls.rar ... CRC = 0xD782FE65 218 219SFV file successfully created (4 files processed) ... 220--- 221 222Let's have a look at the SFV file created: 223 224--- 225; Generated by pdSFV/Unix on Thu Oct 21 19:26:48 1999 226; 227myrls.r00 4C27B20A 228myrls.r01 3BB960FA 229myrls.r02 A01C2C95 230myrls.rar D782FE65 231--- 232 233That's it :) Now for some of the options: 234 235 236Adding logos/banners 237-------------------- 238 239If you have some nice ASCII banner, you can add it on top of the created 240SFV file by using the -l command line option. Let's say your bannerfile 241is /home/lamer/mygroup.nfo, then you would use: 242 243pdsfv -c myrls.sfv -l /home/lamer/mygroup.nfo myrls.r* 244 245to create the SFV file. You will get this confirmation: 246 247--- 248Creating new checksum file: myrls.sfv 249Adding banner file : /home/lamer/mygroup.nfo 250. 251. 252-- 253 254 255Adding "all" files 256------------------ 257 258If called without the -a command line switch, pdSFV only adds *.r?? and *.0?? 259files to the SFV. If you do a 260 261pdsfv -c myrls.sfv -a myrls.r* myrls.nfo 262 263even myrls.nfo's CRC will be stored in the SFV file. Please note that this is 264generally a bad idea to do for various reasons. That's why it's not default and 265you have to use -a if you want to do so. 266 267Also note that you can include myrls.sfv in itself if using -a . This obviously 268is pretty lame, so you might want to try to avoid it :-) 269 270 271WIN-SFV compatibility mode 272-------------------------- 273 274Since still many people are using Winsfv/Sfvnix and as long as there is no 275new version of them out which support SFV files created by other tools, you 276can switch to compatibility mode by using -w ... 277 278pdsfv -c myrls.sfv -w myrls.r* 279 280will then generate something like: 281 282; Generated by WIN-SFV32 v1.5 on Thu Oct 21 19:38:23 1999 283instead of 284; Generated by pdSFV/Unix on Thu Oct 21 19:38:23 1999 285 286You'll get the confirmation: 287 288--- 289Creating new checksum file: myrls.sfv 290Using compatibility mode : Win-SFV 291--- 292 293 294 295TESTING SINGLE FILES 296-------------------- 297 298If you want to verify that one or more single files are good, you can use mode 299-t ... 300 301Examples: 302 303--- 304pdsfv -t myrls.sfv myrls.rar 305 306Testing myrls.rar ... local = 0xD782FE65, listed = 0xD782FE65 - OK 307 3081 file(s) tested - 1 OK - 0 bad... 309--- 310pdsfv -t myrls.sfv myrls.rar myrls.r00 311 312Testing myrls.rar ... local = 0xD782FE65, listed = 0xD782FE65 - OK 313Testing myrls.r00 ... local = 0x8208D936, listed = 0x4C27B20A - BAD 314 3152 file(s) tested - 1 OK - 1 bad... 316--- 317 318You will get errorlevel 0 if all files were good, and errorlevel 1 if 319one or more of the tested files were bad. You can use this in your 320upload check / zipscript stuff. 321 322For glFTPd users: if you modify your zipscript to use pdSFV instead of 323the sfv_check that comes with glFTPd, note that you need command line 324parameters with pdSFV and not just a single filename. Replace 325 326cd $2 327/bin/sfv_check $1 328 329with 330 331cd $2 332/bin/pdsfv -t *.[sS][fF][vV] $1 333 334to get it working! 335 336 337 338TESTING A WHOLE RELEASE 339----------------------- 340 341Use -T to test all the files listed in the sfv file! 342 343--- 344pdsfv -T myrls.sfv 345 346Processing complete check of myrls.sfv ... 347Testing myrls.r00 ... listed = 0x4C27B20A ... local = 0x8208D936 ... BAD 348Testing myrls.r01 ... listed = 0x3BB960FA ... local = 0x3BB960FA ... OK 349Testing myrls.r02 ... listed = 0xA01C2C95 ... local = 0x00000000 ... MISSING 350Testing myrls.rar ... listed = 0xD782FE65 ... local = 0xD782FE65 ... OK 351 3524 file(s) tested - 2 OK - 1 bad - 1 missing ... 353--- 354 355Errorlevels: 0 - all files there and OK 356 1 - one or more files bad 357 2 - one or more files missing 358 359 360 361COUNTING MISSING FILES 362---------------------- 363 364--- 365pdsfv -m myrls.sfv 366 367Performing completion check... 368[total files listed] [local files] [missing files] 3694 3703 3711 372--- 373 374You can use this one in shellscripts to fetch the number of total 375files in this release, the number of how many files you've got 376local yet and how many are missing. 377 378To get the number of files existing locally (3 in our example), you 379could use some shit like: 380 381NUMLOCAL=`pdsfv -m myrls.sfv | tail -n 2 | head -n 1` 382echo "$NUMLOCAL files of this release have arrived yet!" 383 384If you're a funny shellscripter you surely can use this somehow. Just 385note that the files are not tested when using -m ! This means if there 386are some bad files lying around or the files are being uploaded at the 387moment and not finished yet, they will count as "there" and not as 388"missing". 389 390