1@database XpkMaster.guide 2@$VER: XpkMaster.guide 1.70 (08.09.1999) by SDI 3@node Main "Welcome to the XPK distribution" 4Welcome to the eXternal PacKer (XPK) library system for easier handling 5of crunching and decrunching. 6 7 @{"About" LINK About} About this distribution 8 @{"Archive Contents" LINK Contents} Contents of distributed archives 9 @{"Version Info" LINK History} What changes are made 10 @{"GNU - License" LINK GNU-License} License for some of added libs/programs 11 @{"Philosophy" LINK Philosophy} Some background information 12 13 Version 4 news 14 @{"Preferences" LINK Preferences} Preferences system 15 @{"XFD Support" LINK "XFD Support"} Alien cruncher support 16 17 @{"SubLibs" LINK SubLibs} Documentation for supplied sublibraries 18 @{"C-Utils" LINK C-Utils} Documentation of the included programs 19 @{"XPK programs" LINK "XPK programs"} XPK supporting/using programs 20 21 @{"Contacts" LINK Contacts} Addresses of people related to XPK 22 23Try WWW addresses 24 http://www.amigaworld.com/support/xpkmaster/ 25or http://home.pages.de/~Gremlin/xpkmaster.html 26 27Here all files are accessable (also the xpk_Crypt.lha archive). 28On this page is also a list of a lot of XPK sub libraries and XPK programs. 29@endnode 30 31@node Contents "The contents of the distribution" 32xpk_User.lha (util/pack) 33 C/ Various @{"programs" LINK C-Utils} using XPK. 34 catalogs/ Catalog files for use with locale.library. 35 EnvArc Preferences file example. 36 Libs/xpkmaster.library The heart of the XPK system. 37 Libs/compressors/ Compression @{"sublibraries" LINK SubLibs}. 38 Libs_1.3/ xpkmaster.library for OS 1.3. 39 Libs_68020+/ 68020+ versions of some @{"sublibraries" LINK SubLibs}. 40 Prefs/ Preferences program. 41 Install Installer script for complete distribution. 42 XpkMaster.guide This documentation. 43 44xpk_Crypt.lha (not distributed in USA, in Germany util/crypt) 45 libs/compressors/ Encryption @{"sublibraries" LINK SubLibs}. 46 source/ Source files of encryption libs. 47 This archive can be accessed only on German Aminet servers or by calling 48 my WWW page (See below). 49 50xpk_Develop.lha (util/pack) 51 Autodocs/ Programmer documentations of libraries. 52 HotHelp/ HotHelp documentation. 53 Include/ Includes for programmers. 54 55xpk_Source.lha (util/pack) 56 Sources to libraries and utility programs. 57@endnode 58 59@node "XPK programs" 60XFH: (1.40) 61----------- 62XFH-Handler is a DOS handler which uses xpkmaster.library to provide 63transparent access to compressed files in a given directory or partition. 64All compression/decompression is done automatically by the handler when 65files are read or written. Compression is optional and may be switched at 66any time, allowing for fine control over storage of data. The compression 67method may be changed at will. Decompression is always automatic, you 68do not have to care about which compressor was used to create the files. 69 70Author: @{"Matthias Scheler" LINK "Contact Matthias Scheler"} 71Where to find: in Aminet as util/pack/XFH.lha 72 73XPK-KNIGHT: (1.05) 74------------------ 75XPK-KNIGHT is a free configurable, easy graphic user interface for 76comfortable usage of most of the important functions whithin the 77xpkmaster.library and its correspondending XPK packers. 78 79Author: Alexander Grossberger <nobody@betei.franken.de> 80Where to find: in Aminet as util/pack/xpk-Knight_???.lha 81 82XpkArchive System: (2.0) 83------------------------ 84As stated in @{"Philosophy" LINK Philosophy} above the xpkmaster.library there exists also the 85xpkarchive.library, which handles archive creation like LhA or Zoo. Please 86have a look at this package too! 87 88Author: @{"Matthias Meixner" LINK "Contact Matthias Meixner"} 89Where to find: in Aminet as util/arc/XpkArchivePackage??.lha 90 91XPKatana: (1.5) 92--------------- 93Full-featured GUI for XPK, allows (un)packing, features a complete ARexx 94port, supports batch jobs, etc... Also supports FileID.library for 95filetype identification, and xfdmaster.library for alien packer 96decrunching, making it possible to repack most alien packing formats into 97a regular XPK format. 98 99Author: Victor Ducedre <victord@netrover.com> 100Where to find: in Aminet as util/pack/XPKatana.lha 101 102XpkCybPrefs: (1.2) 103------------------ 104XpkCybPrefs is an alternative to XpkMasterPrefs. It provides a transparent 105"intelligent" packing, according to the filetype (by simply setting "USER" 106as XPK packer and configuring once for all, you get everything working). 107Some features: Size conditions, "Multipacking" (trying multiple packers on 108data, keeping the best or fastest-decrunching etc... one!!), report-window 109with full info, user-choice requester ETC ETC... (100% user-configurable) 110 111Author: Alexis Nasr <nasr@hol.fr> 112Where to find: in Aminet as util/pack/XpkCybPrefs.lha 113@endnode 114 115@node About 116This distribution was first created by @{"Dirk St�cker" LINK "Contact Dirk St�cker"} in October 1996. 117 118The concept, the most stuff and documentations of version 2.x distribution 119were made by @{"Bryan Ford" LINK "Contact Bryan Ford"} and @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"}. A lot of bug fixes made 120@{"Christian von Roques" LINK "Contact Christian von Roques"}. I continue XPK development now, because the above 121mentioned persons have no time to do further support. 122 123To the authors of programs supporting XPK: 124 125Please send me a short description (10 lines maximum) of your program. I 126compile a list of XPK supporting programs from this information. 127Please include information, where to find your program (e.g. in Aminet 128util/pack). 129 130About this documentation: 131 132I created this documentation starting from a lot of single files. To cut 133the size a little bit down I had to omit much useless or double 134information. If I deleted something I should not, please tell me and I will 135reincorporate it. Error reports and newer docs or corrected contact 136addresses are welcome too. Some of them are probably outdated. 137 138If you have newer versions of included sub libraries or other enhancements 139please contact me as well. 140 141If someone feels, like translating the Installer script or the catalog 142files for xpkmaster.library and XpkMaster preferences program, send it, and 143I will include the stuff in next release. 144@endnode 145 146@node History 147When the version number has a character attached, the xpkmaster library 148did not change, but only the distribution was updated. 149 1505.2: bug work around for sub librarie errors in Query function 151 1525.1: progress report no longer produces wrong data for really large files 153 1545.0a: bug fix for both preferences programs, additions to xpksub.doc 155autodoc, better debug output for master library, new catal� translation 156 1575.0: added XpkSeek function and a test programs for it, wrote one 158assembler example program on request, bugfix for xpkFAST.library 159 1604.34: xBench has now a LOG option, xpkmaster functions XpkWrite and XpkRead 161now do their job correctly, example programs included, XpkWrite autodoc 162fixed 163 1644.33a: xpkNONE.library had a little bug, which could cause crashes. 165Install script has been fixed a bit as well. 166 1674:33: fixed some little problems and removed access to address 4 (SysBase 168variable is used instead now). This makes the file 400 Bytes larger, but 169hopefully a little bit faster and better designed. 170 1714.32: bug fix (crash) in XpkQuery function (this time not my fault, but a 172compiler error) 173 1744.31: fixed Enforcer hit, new xBench version (better TEST routine), some 175optimizations and bug fixes 176 1774.29: Sorry, sorry, but 4.27 did not really fix the bug. Now it is fixed! 178xpkmaster is now 100% C code (also startup), which results in 100 byte 179longer executable, but it should be portable a lot easier now. 180 1814.27: Fixed serious bug added in last version, with files larger than 18265535 bytes. Thanks to Thomasz Kepa for reporting it. Added missing catalog 183texts. 184 1854.26: Added xfdmaster.library support and cleaned up the code a lot. XPK 186can decrunch StoneCracker, RNC, CrunchMania and Imploder files now. 187The internal PowerPacker support has been removed, as xfdmaster.library 188does this now. So powerpacker.library is no longer required. 189Thanks a lot to Georg H�rmann for making this possible. 190Changed a lot in design of XpkPassRequest. It uses gadtools library and 191is a lot nicer now (has a window title, shows stars, timeout uses 192timer.device, ...). 193 1944.16: Changed A2 contents of preferences RecogFunc to TagList (I hope 195nobody did use it till now.). This causes problems with XpkCybPrefs version 1961.0. Use newer versions only! 197 1984.14a: fixed Installer script, rewrote xDir, added more language files 199 2004.14: fixed low-mem-bug in NUKE and DUKE, little changes in master lib, 201fixed IDEA bug (thanks to Alexis Nasr for finding a solution), fixed bug in 202XpkMasterPrefs, added some parts to the docs, added length check and OR 203specifier to file pattern check, added pattern matching for xBench 204 2054.13: fixed Installer script, added xBench ALL and SAVE option, removed 206error in Expunge code of library, fixed xQuery 207 2084.12: added automatic password request and preferences system, cleared the 209code a bit and removed obsolete code, removed xDrop from distribution, 210wrote new xBench utility. 211 2124.0: added 5 new functions, fixed some little bugs, added first stage of 213preferences system (XpkMaster and XpkMasterPrefs programs), library is now 214OS2.0+ only 215 2163.12: special OS1.3 only version, removed locale and utility library stuff 217 2183.11: recompiled the library with SAS 6.57, cleaned up the complete 219package, removed XFH 220 2213.10: I reworked the source and added locale.library support, a better 222library header and another way of debuging. 223NOTE: The include files moved to xpk directory. Before it was libraries. 224@endnode 225 226@node Preferences "Preferences system overview" 227The preferences system allows packing of files depending on their file 228type, their size or name. It is useful especially for programs handling a 229lot of different files like backup tools. With type depending packing you 230get a faster and shorter backup. It is useful for stacker tools like XFH 231also. But the system may be useful for lots of other situations. 232 233It consists of 2 parts: 234 2351) Program for preferences settings in "Prefs" directory: @{"XpkMaster" LINK "Preferences - XpkMaster"} 236 This program also allows setting @{"global xpkmaster defines" LINK "Preferences - Global"}. 2372) Shell utility to handle preferences files on run-time: @{"XpkMasterPrefs" LINK "Preferences - XpkMasterPrefs"} 238 239* Programmers should read the autodoc file xpkpreferences.doc. There is 240* told how preferences are handled and how you can replace the preferences 241* system with you own one. There already exists another system called 242* XpkCybPrefs. 243 244For the user it is really easy to use: You only need to select the sub 245library 'USER' in the programs which use xpk for packing. This USER library 246is no real library, but a dummy (it does not exist in LIBS:compressors). 247Programs accessing the xpk libraries direct, or scanning LIBS:compressors 248themself are thus not able to use the new preferences! 249 250When preferences are not activated and you select USER you get an error. 251 252To get the preferences active you either need to create a preferences file 253with @{"XpkMaster" LINK "Preferences - XpkMaster"} and to run @{"XpkMasterPrefs" LINK "Preferences - XpkMasterPrefs"} or to install any other system 254like XpkCybPrefs. 255 256The global preferences are used all time when needed (and installed), but 257may be disabled by certain programs. 258 259xpkmaster.library does not access any of the files itself, but only 260accesses the preferences semaphore. xpkmaster.library thus never causes any 261DOS-Requests (except when using file access as source or destination and 262loading sub libraries). That's why it can be used in multitasking 263unfriendly environments like for certain games. 264@endnode 265 266@node "Preferences - XpkMaster" 267It is a normal preferences program like Serial, Printer, Locale, ... 268 269This program is used to set global xpkmaster defines (TimeOut, use of xex 270libraries, ...) and to define file types and corresponding packing methods. 271 272The screen is divided into left and right part. On left side there is a 273list of all filetypes. 4 gadgets allow manipulation of the entries: move up 274and down type, delete type and add new type. First entry always exists and 275cannot be deleted/moved. This is default entry. It's name can be changed, 276but after reloading saved data the default name applies again. When adding 277a new type, the data from default type is copied into new type. 278 279The right side allows manipulation of the filetype: 280 281Name Pattern: 282Specify a standard AmigaDOS pattern describing the filetype. Using field 283File Pattern is better, because some programs do not submit file names or 284use dummy names. It is always better to find a File Pattern description, 285than a Name Pattern. 286 287File Pattern: 288This allows to describe a file type by comparing the file contents with a 289defined pattern. See @{"filepatterns" LINK Filepatterns} for pattern description and examples. 290 291Library Name: 292Four letter ID of library you want use for packing. 293 294Mode: 295Mode used for packing. This is a decimal number from 0 to 100. 296 297ChunkSize: 298ChunkSize used for packing. See autodoc files for better description. Most 299time it should be 0. 300 301PackMode: 302Define if you want to pack the file, return an error or do not pack the 303file. When selected 'return error', the caller program gets told, that 304packing was not possible. When selecting 'do not pack', the file is passed 305through without changes. If these two type are selected, Library name, Mode 306and ChunkSize fields are disabled. 307Modus 'do not pack' may produce errors on decrunching with certain backup 308programs. Try using 'return error' for these instead. If this does not 309work, you cannot use any of these two (for example in Diavolo backup), but 310only normal type selection. You may choose packer "NONE" for the previous 311NoPack/ReturnError entries. 312 313For the last part, see @{"global defines" LINK "Preferences - Global"}. These defines are not file type 314specific, but xpkmaster global. 315 316The last three gadgets (Save, Use, Cancel) and the menus are equal to 317standard prefs files and easy understandable. 318 319The patterns: When both pattern types are specified, both must be true, 320to specify a filetype. 321 322Example: GIF files 3231) filepattern and no namepattern is specified: 324 XpkMaster recognizes all GIF files 3252) filepattern and name pattern (#?.gif) is specified: 326 XpkMaster recognizes all GIF files, ending with .gif 3273) on name pattern is specified: 328 XpkMaster recognizes all files ending with #?.gif (also if these files 329 are not GIF files!) 330 331A correct file pattern should most times be enough and better as a name 332pattern. So example 1 is nearly in all cases the best way! 333@endnode 334 335@node "Preferences - Global" 336These settings allow to define some global xpkmaster.library behaviour. All 337different preferences systems should support these defines. 338 339@{"Use XFD:" LINK "XFD Support"} 340Should xpkmaster use xfdmaster.library for decrunching custom file types, 341or not? Not all xfdmaster file types are supported. XPK can decrunch only 342types, which support calculation of uncrunched file size before 343uncrunching. This is turned off by default, but I suggest to turn it on. 344 345Use XEX: 346Should xpkmaster install list, where the scan routines of xex libraries 347are hold. This is necessary to recognize custom filetypes on decrunching. 348It is not imlemented at the moment. 349 350Auto Password: 351Should xpkmaster use automatic password requester, when password is needed, 352but not supplied? Password requester quits after a definable time value. 353 354Timeout: 355Define the time password request stays open with no user interaction. After 356that time the requester automatically quits, when no user-action happend. 357Default is 120 seconds. 358@endnode 359 360@node "Preferences - XpkMasterPrefs" 361This program is like IPrefs. It loads the file ENV:xpkmaster.prefs and 362makes it possible for xpkmaster.library to access the data. How this is 363done is explained in the autodoc file xpkpreferences.doc. Everytime you 364change the file ENV:xpkmaster.prefs by selecting USE or SAVE in 365XpkMaster preferences program, the data is updated automatically. So after 366selecting USE, xpkmaster.library uses the new settings. 367 368NOTE: The filetypes are scanned in same order, as you entered them in 369XpkMaster preferences program. There is only one exception: The default 370type (displayed first always) is not scanned, but used when no other type 371matches (which means it is the last always!). 372 373How to install: 374Copy it to C: directory and start the program before you use xpkmaster 375library first. The best place is S:User-StartUp (or S:StartUp-Sequence in 376some cases). There you may add the line 377 378 Run >NIL: XpkMasterPrefs 379 380The rest will be done automatically by the program. To kill the program 381call it with "XpkMasterPrefs QUIT" or send it a CTRL-C break signal. 382 383You can start it from WorkBench (or from drawer WBStartUp) as well. You 384only need to add a icon and set the tooltype "DONOTWAIT". But quitting the 385program from WorkBench is impossible. You need to enter shell for that and 386use above command line. 387@endnode 388 389@node Filepatterns 390The file pattern consists of a mode describing lower case character 391(l, h, m, v, r, g, s) and corresponding hexadecimal data. It helps to 392describes file type specific parts. 393You may specify different types in one string by seperating the types by 394'|'. So text1|text2 means that the file has to match either text1 or text2. 395 396Control Characters: 397 398For these control caracters the data consists of a $/hex value with upto 3998 digits (ULONG) [option m allows only 4 digits (UWORD)]. 400 401l - filesize must be shorter (LOWER) than specified value 402h - filesize must be larger (HIGHER) than specified value 403 When using these options, be sure you do not forget any size. When 404 specifying a type l100 and one type h100, the size 100 is missing! 405 406m - Move to position in the file. The first byte has position 0. 407 Note: The real maximum move-size depends on the buffer value set in 408 XpkMasterPrefs program. Currently this match part are the first 2048 409 bytes. 410 411The following numbers/strings are specified using $/hex values and must be 412given as bytes (2 digits). You may repeat data behind these without 413adding the command each time (f.e. v40v41 is equal to v4041). 414 415v - Match specified BYTES at the current position in the buffer. 416r - Exclude ranges - these ranges cannot be within the match part of the 417 file. You can once again specify numerous ranges, all will be compared 418 against the first 2048 bytes. For example, to figure out an plain text 419 file you might try r001F7F9F, so any file with chars within 00-1F and 420 7F-9F is passed through to the next preference entry. 421 You need to specify always 2 bytes (4 digits). 422g - Range which may exists at the current position - means same as 423 v but gives a valid range instead. For example, g3039 will match 424 any numeral digit at the current position. 425 You need to specify always 2 bytes (4 digits). 426s - Describes a string which has to be within the match part of the file, 427 but at any given position therein. For example, you could detect post- 428 script files using s25215053 ("%!PS"). 429 430examples: 431v464F524D describes an IFF file - matching FORM at the start 432v464F524Dm8v494C424D describes an IFF ILBM file - matching FORM at the 433 start and ILBM at position 8 434r00405BFF matches any file having only UPPER characters A-Z 435g415A617A describes a file having an UPPER character at 436 first and a lower character at second position 437v000003F3l6401 Executable file with size up to 25KB 438v000003F3h6400l19001 Executable file with size between 25KB and 100KB 439v000003F3h19000 Executable file with size greater than 100KB 440v50503230|v50583230 PowerPacker files: Header is PP20 or PX20 441 442Example File Types: 443 444Files marked with NoPack are already crunched. 445 446Type NoPack File Pattern 447 448DMS Archive (DMS) * v444D5321 449Executable (EXE) v000003F3 450GIF Picture (GIF) * v474946 451HotHelpHeader (HHH) m6v486F7448656C70486561646572 452HotHelpText (HHT) * m6v486F7448656C7054657874 453Icon File (INFO) vE3100001 454ILBM Picture (IFF ILBM) v464F524Dm8v494C424D 455Intellifont (IOF) v00440001m10v0014FFFF 456JPEG Picture (JFIF) * vFFD8 457MED Module (MMD) v4D4D44g3039 458LhA Archive (-lh?-) * m2v2D6C68m6v2D 459LZX Archive (LZX) * v4C5A58 460PNG Picture (PNG) * v89504E470D0A1A0A 461PT 32 Module (PT MOD 32) m438v4D2E4B2E 462PT Packed Song (PT SONG) * v5041434B 463Sound File (IFF 8SVX) v464F524Dm8v38535658 464Startrek. Mod. (ST MOD) m438v464C54g3039 465WordWorth Text (IFF WOWO) v464F524Dm8v574F574F 466XPK file (XPKF) * v58504B46 467Zip Archive (PK) * v504B0304 468Other IFF (IFF) v464F524D 469@endnode 470 471@node "XFD Support" 472Starting with version 4.19 of xpkmaster.library, it is posible to decrunch 473files, which are not in XPKF file format (not crunched with XPK). 474 475XPK uses the shared xfdmaster.library to decrunch these files. This library 476can be found in Aminet (util/pack/xfd???.lha). You need at least version 477V38 of this library. 478 479To enable that feature you need a program, which allows this by default or 480you use @{"preferences" LINK Preferences} system to turn it on. By default the XFD support is 481turned off. 482 483Not all xfdmaster file types are supported. XPK can decrunch only types, 484which support calculation of uncrunched file size before uncrunching. 485 486Version 38.2 of XFD allows to decrunch ProPack (RNC), StoneCracker, 487CrunchMania, Imploder and PowerPacker data. 488@endnode 489 490@node Philosophy "XPK - A STANDARD FOR DATA COMPRESSION" 491MOTIVATION 492---------- 493* Many programs that should offer data compression (e.g. HD backup 494 utilities) do not. 495* Many programs that offer data compression use old, slow, inefficient or 496 inappropriate algorithm. 497* All programs that offer data compression offer just one algorithm, you 498 are stuck with that one. 499* Many good packers are not used by any application program and have no 500 good user interface. 501* The installation of most packers requires AmigaShell knowledge (f.e. 502 putting LhA in the path so that other programs can find it) 503* The decompression of all files packed with existing packers requires 504 knowledge about the packer used for compression. 505* Many compression programs can not deal with files that are larger than 506 available memory. 507* The existing compression programs are either slow or have a low 508 compression factor. 509* There is no way to support upcoming hardware compression cards in already 510 existing applications yet. 511* For none of the current compression programs exists a real decompressing 512 file handler that uses no dirty tricks to decompress files on the fly. 513 514The solution to all these problems is XPK. 515 516OVERVIEW 517-------- 518The XPK standard is to data compression what xpr is to file transmission. 519It consists of three layers: 520Level 2: The application/XPK interface for archives 521Level 1: The application/XPK interface for files 522Level 0: The XPK/packer interface 523In addition, there is an optional standard XPK file format. 524 525All parts of the XPK standard are implemented in shared libraries. There 526is one master library for archive level access, one master library for file 527level access, and one library for each compression algorithm. 528 529level 2 xpkarchive.library 530 | | 531 V | 532level 1 xpkmaster.library | 533 | | | V 534level 0 type 3 | | | xarZOO 535 | | V 536 type 2 | | xexXPWP 537 V V 538 type 1 xpkNUKE xpkENCO 539 540LEVEL 0 LIBRARIES 541----------------- 542All level 0 libraries offer the same functions. They are very small. 543Typical calls are: "Tell me what you can", "Compress this chunk of memory 544to another chunk of memory", and "Decompress this chunk of memory to 545another chunk of memory". These libs are very limited, their functionality 546is expanded by xpkmaster.library. No one would want to talk to a sub 547library directly. 548 549THE LEVEL 1 LIBRARY 550------------------- 551Offers functions like "Compress this file to that chunk of memory using 552that algorithm". All combinations permitted: Mem to mem, file to file, mem 553to file, decompression and compression. Asynchronous packing possible. Very 554convenient tag based caller interface. Determines automatically out which 555sub library to use for decompression. Returns detailed error messages. 556 557THE LEVEL 2 LIBRARY 558------------------- 559Offers archiving functions like "add this file to that archive" or "show 560me the contents of that archive". 561 562OVERRIDING 563---------- 564It is planned, that libraries of a lower level can offer higher level 565functions. They should be able to override the automatic functionality 566expansion by the higher level library. xpkmaster.library, for example, 567enforces the use of the XPK standard file format. It should be possible to 568override this by a sub library. Therefore an new library interface will be 569created, the xex libraries. 570 571XPK also supports to decrunch alien data cruncher formats by use of the 572@{"xfdmaster.library" LINK "XFD support"}. But not all of xfdmaster's file types are supported. 573XPK can decrunch only types, which support calculation of uncrunched file 574size before uncrunching. 575 576THE XPK FILE FORMAT 577------------------- 578Offers checksums, chunks (important when Seek()s on a compressed file 579become necessary) and automatic handling by the xpkmaster.library. This 580means that any new packer that can only pack mem to mem has its own file 581format immediately. And most important: The name of the packer library is 582contained in the file. Therefore, copying a new sub library to LIBS: is 583all you have to do to install a new packer (easily done in installation 584scripts); xpkmaster.library recognizes the new file type immediately. No 585changes to xpkmaster.library or the application programs necessary. In 586case the XPK file format is not used, the introduction of a new packer 587requires a change the xpkmaster.library. 588 589TYPICAL APPLICATIONS 590-------------------- 591A few examples for applications that could use XPK: 592* A GadTools based archiver interface that can deal with all archivers 593* A CLI based file compressor/decompressor [xPack, xPK, xUp] 594* A hard disk backup utility that stores compressed data [Diavolo and 595 others] 596* A tool to write compressed images of devices to files [PackDev] 597* A 'more' program with automatic decompression [Most, MuchMore] 598* A DTP program that stores its fonts in compressed format 599* A network protocol with built in data compression for slow connections 600* A hypertext utility that allows all data to be compressed 601* A file handler that overlays an existing filesystem and uncompresses any 602 file while loading [XFH, PackDisk, Diskexpander (EPU)] 603...plus many more we do not even need to think about yet. 604 605CONCLUSION 606---------- 607XPK would increase the usefulness and flexibility of both application and 608compression programs while improving their user friendliness at the same 609time. The best way to establish this standard would have been to distribute 610it on the workbench disk that came with every Amiga. 611@endnode 612 613@node GNU-License 614 GNU GENERAL PUBLIC LICENSE 615 Version 2, June 1991 616 617Copyright (C) 1989, 1991 Free Software Foundation, Inc. 618 675 Mass Ave, Cambridge, MA 02139, USA 619Everyone is permitted to copy and distribute verbatim copies 620of this license document, but changing it is not allowed. 621 622 Preamble 623 624The licenses for most software are designed to take away your 625freedom to share and change it. By contrast, the GNU General Public 626License is intended to guarantee your freedom to share and change free 627software--to make sure the software is free for all its users. This 628General Public License applies to most of the Free Software 629Foundation's software and to any other program whose authors commit to 630using it. (Some other Free Software Foundation software is covered by 631the GNU Library General Public License instead.) You can apply it to 632your programs, too. 633 634When we speak of free software, we are referring to freedom, not 635price. Our General Public Licenses are designed to make sure that you 636have the freedom to distribute copies of free software (and charge for 637this service if you wish), that you receive source code or can get it 638if you want it, that you can change the software or use pieces of it 639in new free programs; and that you know you can do these things. 640 641To protect your rights, we need to make restrictions that forbid 642anyone to deny you these rights or to ask you to surrender the rights. 643These restrictions translate to certain responsibilities for you if you 644distribute copies of the software, or if you modify it. 645 646For example, if you distribute copies of such a program, whether 647gratis or for a fee, you must give the recipients all the rights that 648you have. You must make sure that they, too, receive or can get the 649source code. And you must show them these terms so they know their 650rights. 651 652We protect your rights with two steps: (1) copyright the software, and 653(2) offer you this license which gives you legal permission to copy, 654distribute and/or modify the software. 655 656Also, for each author's protection and ours, we want to make certain 657that everyone understands that there is no warranty for this free 658software. If the software is modified by someone else and passed on, we 659want its recipients to know that what they have is not the original, so 660that any problems introduced by others will not reflect on the original 661author's reputations. 662 663Finally, any free program is threatened constantly by software 664patents. We wish to avoid the danger that redistributors of a free 665program will individually obtain patent licenses, in effect making the 666program proprietary. To prevent this, we have made it clear that any 667patent must be licensed for everyone's free use or not licensed at all. 668 669The precise terms and conditions for copying, distribution and 670modification follow. 671 672 GNU GENERAL PUBLIC LICENSE 673 TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 674 6750. This License applies to any program or other work which contains 676a notice placed by the copyright holder saying it may be distributed 677under the terms of this General Public License. The "Program", below, 678refers to any such program or work, and a "work based on the Program" 679means either the Program or any derivative work under copyright law: 680that is to say, a work containing the Program or a portion of it, 681either verbatim or with modifications and/or translated into another 682language. (Hereinafter, translation is included without limitation in 683the term "modification".) Each licensee is addressed as "you". 684 685Activities other than copying, distribution and modification are not 686covered by this License; they are outside its scope. The act of 687running the Program is not restricted, and the output from the Program 688is covered only if its contents constitute a work based on the 689Program (independent of having been made by running the Program). 690Whether that is true depends on what the Program does. 691 6921. You may copy and distribute verbatim copies of the program's 693source code as you receive it, in any medium, provided that you 694conspicuously and appropriately publish on each copy an appropriate 695copyright notice and disclaimer of warranty; keep intact all the 696notices that refer to this License and to the absence of any warranty; 697and give any other recipients of the Program a copy of this License 698along with the Program. 699 700You may charge a fee for the physical act of transferring a copy, and 701you may at your option offer warranty protection in exchange for a fee. 702 7032. You may modify your copy or copies of the Program or any portion 704of it, thus forming a work based on the Program, and copy and 705distribute such modifications or work under the terms of Section 1 706above, provided that you also meet all of these conditions: 707 708a) You must cause the modified files to carry prominent notices 709stating that you changed the files and the date of any change. 710 711b) You must cause any work that you distribute or publish, that in 712whole or in part contains or is derived from the Program or any 713part thereof, to be licensed as a whole at no charge to all third 714parties under the terms of this License. 715 716c) If the modified program normally reads commands interactively 717when run, you must cause it, when started running for such 718interactive use in the most ordinary way, to print or display an 719announcement including an appropriate copyright notice and a 720notice that there is no warranty (or else, saying that you provide 721a warranty) and that users may redistribute the program under 722these conditions, and telling the user how to view a copy of this 723License. (Exception: if the Program itself is interactive but 724does not normally print such an announcement, your work based on 725the Program is not required to print an announcement.) 726 727These requirements apply to the modified work as a whole. If 728identifiable sections of that work are not derived from the Program, 729and can be reasonably considered independent and separate works in 730themselves, then this License, and its terms, do not apply to those 731sections when you distribute them as separate works. But when you 732distribute the same sections as part of a whole which is a work based 733on the Program, the distribution of the whole must be on the terms of 734this License, whose permissions for other licenses extend to the 735entire whole, and thus to each and every part regardless of who wrote it. 736 737Thus, it is not the intent of this section to claim rights or contest 738your rights to work written entirely by you; rather, the intent is to 739exercise the right to control the distribution of derivative or 740collective works based on the Program. 741 742In addition, mere aggregation of another work not based on the Program 743with the Program (or with a work based on the Program) on a volume of 744a storage or distribution medium does not bring the other work under 745the scope of this License. 746 7473. You may copy and distribute the Program (or a work based on it, 748under Section 2) in object code or executable form under the terms of 749Sections 1 and 2 above provided that you also do one of the following: 750 751a) Accompany it with the complete corresponding machine-readable 752source code, which must be distributed under the terms of Sections 7531 and 2 above on a medium customarily used for software interchange; 754or, 755 756b) Accompany it with a written offer, valid for at least three 757years, to give any third party, for a charge no more than your 758cost of physically performing source distribution, a complete 759machine-readable copy of the corresponding source code, to be 760distributed under the terms of Sections 1 and 2 above on a medium 761customarily used for software interchange; or, 762 763c) Accompany it with the information you received as to the offer 764to distribute corresponding source code. (This alternative is 765allowed only for noncommercial distribution and only if you 766received the program in object code or executable form with such 767an offer, in accord with Subsection b above.) 768 769The source code for a work means the preferred form of the work for 770making modifications to it. For an executable work, complete source 771code means all the source code for all modules it contains, plus any 772associated interface definition files, plus the scripts used to 773control compilation and installation of the executable. However, as a 774special exception, the source code distributed need not include 775anything that is normally distributed (in either source or binary 776form) with the major components (compiler, kernel, and so on) of the 777operating system on which the executable runs, unless that component 778itself accompanies the executable. 779 780If distribution of executable or object code is made by offering 781access to copy from a designated place, then offering equivalent 782access to copy the source code from the same place counts as 783distribution of the source code, even though third parties are not 784compelled to copy the source along with the object code. 785 7864. You may not copy, modify, sublicense, or distribute the Program 787except as expressly provided under this License. Any attempt 788otherwise to copy, modify, sublicense or distribute the Program is 789void, and will automatically terminate your rights under this License. 790However, parties who have received copies, or rights, from you under 791this License will not have their licenses terminated so long as such 792parties remain in full compliance. 793 7945. You are not required to accept this License, since you have not 795signed it. However, nothing else grants you permission to modify or 796distribute the Program or its derivative works. These actions are 797prohibited by law if you do not accept this License. Therefore, by 798modifying or distributing the Program (or any work based on the 799Program), you indicate your acceptance of this License to do so, and 800all its terms and conditions for copying, distributing or modifying 801the Program or works based on it. 802 8036. Each time you redistribute the Program (or any work based on the 804Program), the recipient automatically receives a license from the 805original licensor to copy, distribute or modify the Program subject to 806these terms and conditions. You may not impose any further 807restrictions on the recipient's exercise of the rights granted herein. 808You are not responsible for enforcing compliance by third parties to 809this License. 810 8117. If, as a consequence of a court judgment or allegation of patent 812infringement or for any other reason (not limited to patent issues), 813conditions are imposed on you (whether by court order, agreement or 814otherwise) that contradict the conditions of this License, they do not 815excuse you from the conditions of this License. If you cannot 816distribute so as to satisfy simultaneously your obligations under this 817License and any other pertinent obligations, then as a consequence you 818may not distribute the Program at all. For example, if a patent 819license would not permit royalty-free redistribution of the Program by 820all those who receive copies directly or indirectly through you, then 821the only way you could satisfy both it and this License would be to 822refrain entirely from distribution of the Program. 823 824If any portion of this section is held invalid or unenforceable under 825any particular circumstance, the balance of the section is intended to 826apply and the section as a whole is intended to apply in other 827circumstances. 828 829It is not the purpose of this section to induce you to infringe any 830patents or other property right claims or to contest validity of any 831such claims; this section has the sole purpose of protecting the 832integrity of the free software distribution system, which is 833implemented by public license practices. Many people have made 834generous contributions to the wide range of software distributed 835through that system in reliance on consistent application of that 836system; it is up to the author/donor to decide if he or she is willing 837to distribute software through any other system and a licensee cannot 838impose that choice. 839 840This section is intended to make thoroughly clear what is believed to 841be a consequence of the rest of this License. 842 8438. If the distribution and/or use of the Program is restricted in 844certain countries either by patents or by copyrighted interfaces, the 845original copyright holder who places the Program under this License 846may add an explicit geographical distribution limitation excluding 847those countries, so that distribution is permitted only in or among 848countries not thus excluded. In such case, this License incorporates 849the limitation as if written in the body of this License. 850 8519. The Free Software Foundation may publish revised and/or new versions 852of the General Public License from time to time. Such new versions will 853be similar in spirit to the present version, but may differ in detail to 854address new problems or concerns. 855 856Each version is given a distinguishing version number. If the Program 857specifies a version number of this License which applies to it and "any 858later version", you have the option of following the terms and conditions 859either of that version or of any later version published by the Free 860Software Foundation. If the Program does not specify a version number of 861this License, you may choose any version ever published by the Free 862Software Foundation. 863 86410. If you wish to incorporate parts of the Program into other free 865programs whose distribution conditions are different, write to the author 866to ask for permission. For software which is copyrighted by the Free 867Software Foundation, write to the Free Software Foundation; we sometimes 868make exceptions for this. Our decision will be guided by the two goals 869of preserving the free status of all derivatives of our free software and 870of promoting the sharing and reuse of software generally. 871 872 NO WARRANTY 873 87411. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 875FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 876OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 877PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 878OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 879MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 880TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 881PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 882REPAIR OR CORRECTION. 883 88412. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 885WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 886REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 887INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 888OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 889TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 890YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 891PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 892POSSIBILITY OF SUCH DAMAGES. 893 894 END OF TERMS AND CONDITIONS 895 896 Appendix: How to Apply These Terms to Your New Programs 897 898If you develop a new program, and you want it to be of the greatest 899possible use to the public, the best way to achieve this is to make it 900free software which everyone can redistribute and change under these terms. 901 902To do so, attach the following notices to the program. It is safest 903to attach them to the start of each source file to most effectively 904convey the exclusion of warranty; and each file should have at least 905the "copyright" line and a pointer to where the full notice is found. 906 907<one line to give the program's name and a brief idea of what it does.> 908Copyright (C) 19yy <name of author> 909 910This program is free software; you can redistribute it and/or modify 911it under the terms of the GNU General Public License as published by 912the Free Software Foundation; either version 2 of the License, or 913(at your option) any later version. 914 915This program is distributed in the hope that it will be useful, 916but WITHOUT ANY WARRANTY; without even the implied warranty of 917MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 918GNU General Public License for more details. 919 920You should have received a copy of the GNU General Public License 921along with this program; if not, write to the Free Software 922Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. 923 924Also add information on how to contact you by electronic and paper mail. 925 926If the program is interactive, make it output a short notice like this 927when it starts in an interactive mode: 928 929Gnomovision version 69, Copyright (C) 19yy name of author 930Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type 931'show w'. This is free software, and you are welcome to redistribute it 932under certain conditions; type 'show c' for details. 933 934The hypothetical commands 'show w' and 'show c' should show the appropriate 935parts of the General Public License. Of course, the commands you use may 936be called something other than 'show w' and 'show c'; they could even be 937mouse-clicks or menu items--whatever suits your program. 938 939You should also get your employer (if you work as a programmer) or your 940school, if any, to sign a "copyright disclaimer" for the program, if 941necessary. Here is a sample; alter the names: 942 943Yoyodyne, Inc., hereby disclaims all copyright interest in the program 944`Gnomovision' (which makes passes at compilers) written by James Hacker. 945 946<signature of Ty Coon>, 1 April 1989 947Ty Coon, President of Vice 948 949This General Public License does not permit incorporating your program into 950proprietary programs. If your program is a subroutine library, you may 951consider it more useful to permit linking proprietary applications with the 952library. If this is what you want to do, use the GNU Library General 953Public License instead of this License. 954@endnode 955 956@node SubLibs "Documentation of the included sub libraries" 957 958 @{"CBR0" LINK CBR0} Yet another CmpByteRun0 algorithm compressor 959 @{"DLTA" LINK DLTA} A trivial delta preprocessor 960 @{"DUKE" LINK DUKE} A @{"NUKE" LINK NUKE} variant tuned for sampled sound 961 @{"FAST" LINK FAST} A fast LZRW based compression algorithm 962 @{"FEAL" LINK FEAL} A Fast Encryprion ALgorithm 963 @{"HFMN" LINK HFMN} A fast packing & unpacking dynamic huffman 964 @{"HUFF" LINK HUFF} A dynamic huffman cruncher/decruncher 965 @{"IDEA" LINK IDEA} ABPs IDEA implementation for XPK 966 @{"IMPL" LINK IMPL} A LZ77 variant supporting various compression modes 967 @{"MASH" LINK MASH} Another LZRW based compression algorithm 968 @{"NONE" LINK NONE} A dummy packer doing no compression 969 @{"NUKE" LINK NUKE} A LZ77 variant with fast decompression 970 @{"RAKE" LINK RAKE} A cruncher of the LZ77 family 971 @{"SHRI" LINK SHRI} LZARI variant 972 @{"SMPL" LINK SMPL} A dynamic huffman with delta precoding 973 @{"SQSH" LINK SQSH} A LZ based cruncher with special algorithms for 8 bit sample data 974 975Use @{"xBench" LINk xBench} for comparission between packers. 976 977Legal issues: 978These libraries may be freely distributed, as long as they are kept in 979their original, complete, and unmodified form. It may not be distributed 980by itself or in a commercial package of any kind without a written 981permission. 982 983These libraries are distributed in the hope that they will be useful, but 984WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY 985or FITNESS FOR A PARTICULAR PURPOSE. 986 987Most of them you can redistribute and/or modify under the terms of the 988@{"GNU General Public License" LINK GNU-License}. 989@endnode 990 991@node CBR0 992@toc SubLibs 993 Copyright 1992 Bilbo the first of Hypenosis 994 995xpkCBR0.library is a standard XPK sublibrary implementing the very simple 996cmp byte run 0 compression algorithm. The same algorithm is used on 997compressed IFF-ILBM files. It is well known that this algorithm is only 998efficient on data containing repeating equal bytes. This means that ASCII 999files or (not compressed) picture files will be compressed well, but 1000executable files, sound data files, encrypted files (or other white noise 1001data) will be compressed only approx. 3%. 1002 1003xpkCBR0.library is of course: 1004 1005 � written 100% in 68000 assembler using DevPac V3.02, 1006 � reentrant, 1007 � pc-relative (except for resident structure used by system for injection), 1008 � some bytes shorter than @{"U.D.M�ller's" LINK "Contact Urban Dominik M�ller"} RLEN, 1009 � 2.9 times faster on compression, 3.6 times faster on decompression 1010 compared to RLEN both used on file AmigaVision 1011 � written by Bilbo the first of Hypenosis (this fact should convince you) 1012 1013Version History 10141.0 First public release. 1015 No known bugs. 1016@endnode 1017 1018@node DLTA 1019@toc SubLibs 1020FEATURES 1021-good when crunching modules/sounds in combination with a cruncher 1022-written in fast optimized assembly (joh mei) 1023 1024DOCUMENTATION 1025The DELTA enciphering routines of xpkDLTA were developed to help 1026XPK-crunchers crunching SOUNDS and MODULES. 1027 1028In this version xpkDLTA supports BYTE-Delta encoding. This is the most 1029efficient encoding algorithm in combination with samples. WORD-Delta and 1030LONG-Delta may follow if the first 16-BIT (32-BIT!?) samples pass my way 1031or if you ask me kindly. 1032 1033Example: 10348SVX-Sample, Music mixed with talking 1035 1036Uncrunched: 1016484 byte 8SVX-Sample 1037 1038Imploded: 789824 byte (77.7% left) IMPL.100ed 8SVX 1039 1040Deltaed+Imploded: 628076 byte (61.7% left) DELTA+IMPL.100ed 8SVX 1041 1042HOW IT WORKS 1043------------ 1044xpkDLTA takes a byte and looks what the difference is between this 1045byte and its successor. It stores these differences. That is all! 1046 1047Example: 1048 1049|DATA |DELTA 1050|-------+----- 1051|6 |+6 (6-0) ;Start-Value => precedessor=NULL 1052|7 |+1 (7-6) 1053|3 |-4 (3-7) 1054|4 |+1 (4-3) 1055|10 |+6 (10-4) 1056 1057FUNCTIONS: xpkDLTA supports all standard XPK sublibrary functions. 1058 1059CONTACT 1060If you want an update, enclose enough DM (Deutsche Mark) for disk, stamps, 1061envelope etc. 1062 1063Never forget to mention 1064-what of my programs you are using 1065-which version 1066-where you got it from 1067 1068Fanpost, donations, suggestions, ideas, flames & comments are welcome. 1069 1070@{"Get the authors address (Stephan Fuhrmann)." LINK "Contact Stephan Fuhrmann"} 1071@endnode 1072 1073@node DUKE 1074@toc SubLibs 1075DUKE is a hacked version of @{"NUKE" LINK NUKE} combining the effects of @{"DLTA" LINK DLTA} and @{"NUKE" LINK NUKE}. 1076Its compression performance and ratio probably is not good enough, we 1077still need a good lossless sound and/or module packer. 1078@endnode 1079 1080@node FAST 1081@toc SubLibs 1082xpkFAST is an XPK compression sublibrary whose main purpose is to be 1083fast. The most interesting part of FAST is its speedy compressor, which 1084makes it predestined for applications which compress about as often as they 1085decompress. Good examples are: backup systems which make use of XPK to 1086support compressed backups or compressing filesystems. 1087 1088FAST consists of three parts, two compressors and a common decompressor. 1089You can choose between the two compressors by using FAST.0 up to FAST.79 1090for the ``speedy'' compressor and FAST.80 up to FAST.100 for the 1091``crawling'' compressor, which is still faster than @{"NUKE" LINK NUKE}. The default mode 1092is FAST.50 which selects the ``speedy'' compressor. 1093 1094 Algorithm 1095 --------- 1096 1097FAST is a member of the LZ77 family of datacompressors. Other popular 1098members of the LZ77 family are: xpkNUKE, PowerPacker, Imploder (xpkIMPL) 1099and some parts of lha, gzip, zip, zoo, freeze, arj, uc2, ha, ain, ... 1100 1101The common thing about all LZ77 compressors is that they store the data 1102as sequences of <copy>- and <quote>-items. FAST uses one `control-bit' to 1103distinguish between a <copy>- and a <quote>-item. A <quote>-item simply 1104consists of one byte which has to be placed into the outputstream 1105uninterpreted. Each <copy>-item consists of 12 bit <distance>- and 4 bit 1106<length>-information. <distance> encodes where to copy _from_. The 4095 1107useful possibilities are 1..4095(*) bytes back in the outputstream. 1108<length> encodes _how_many_ bytes to copy. Possible <length>s range from 3 1109to 18, which are encoded as 18-<length>. 1110 1111The input: aaaaadadada compresses to: Q(a) C(1,4) Q(d) C(2,5). Where 1112Q(char) is a <quote>-item and means write a single character 'char' to the 1113output and the <copy>-item C(dist,len) means copy 'len' bytes, which can be 1114found 'dist' bytes back in the output, to the output. 1115 1116FAST uses two datastreams. That is, the compressed data consists of two 1117parts, the wordstream and the bytestream. The first compressor which used 1118this technique was xpkNUKE. The bytestream starts at the beginning of the 1119compressed data and the wordstream is stored in reverse order beginning at 1120the end of the compressed data. Thus the compressed data does look like 1121this: literalsSSDDRROOWW where small characters denote literal bytes and 1122two capital characters are a word from the wordstream. 1123 1124If you want to discover more of the internal workings of xpkFAST just: 1125``Use the force! Read the source!'' The best place to start your tour 1126through the source is the decompressor in decompress.s since the 1127decompressor is much simpler than the compressor. 1128 1129(*) I could have been using distances of 1..4096, but doing so would 1130 have added one instruction to the short and thus fast decompressor. 1131 1132History: 1133 1134In April 1991, Ross Williams published his LZRW1 algorithm by presenting it 1135at the data compression conference. 1136 1137The LZRW1-A algorithm is a direct descendant of the LZRW1 algorithm, 1138improving it a little in both speed and compression. 1139 1140FAST started as a ``port'' of Ross Williams' LZRW1-A C-Implementation 1141and his 68000-version of the decompressor to the Amiga as XPK sublibrary. 1142While porting I made some small changes improving the decompression speed. 1143I removed the feature of handling the case of noncompressable input, 1144because the xpkmaster.library takes care of that. After that, I found some 1145cute changes which dramatically improved the speed of the decompressor. 1146These were in detail: 1147 1148 * split the compressed data into a word- and a bytestream, removing many 1149 double byte accesses with a shift in between. 1150 * changed the copy loop from a move-dbra loop to 18 moves in a row. 1151 * changed the used range from 1..4096 to 0..4095 eliminating one 1152 instruction in the decompression loop. 1153 * removed all bra.s from the inner decompression loop. 1154 * totally rewrote the compressor in 68000 assembler. 1155 + changed the hashfunction to NOT use mul or div. 1156 + produces the ``new'' format needed by the new decompressor. 1157 + removed nearly all of the loop control tests by having 1158 a fast and a safe loop. 1159 + small code fits into the instructioncache of a 68030. 1160 1161 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} helped me to improve the speed of the compressor 1162even further, contributing several ideas and some code. For details refer 1163to the source. 1164 1165V1.00: release date: 29-Aug-1993 1166V1.01: unreleased. [testversion with four different compressors.] 1167V1.02: release date: 12-Sep-1993 1168 * quadrupled the HASHSIZE for FAST.80 .. FAST.100 which allowed the 1169 removal of 2 now unneeded COMPARE_BYTEs to speed up compress_slow. 1170 1171V1.03: release date: 17-Oct-1993 1172 * major code juggling in compress2.s to squeeze some cycles. 1173 * removed the need for a ctrlCtr in compress2.s in favour of doing 1174 addx.w ctrl,ctrl bcs.s ctrlFull and ctrl initialized to #1 1175 instead of rol.w #1,ctrl dbra ctrlCnt,notFull and ctrl initialized 1176 to #$0000FFFF 1177V1.04: release date: 06-Feb-1994 1178 * fixed a buglette reported by Detlef Riekenberg <eule@netgate.fido.de> 1179V1.05: release date: 01-May-1994 1180 * removed MEMF_CLEAR from call to AllocMem() of the hashtable which 1181 is initialised anyway. reported by Simone Avogadro 1182 * cosmetic changes to compress2.s 1183V1.06: release date: 28-Jul-1994 1184 * tuned the copying of the wordstream in compress.s and compress2.s 1185 * rewrote bitreading in the decompressor 1186 1187"Thank you"s must go to: 1188 J�rg Bublath <bublath@forwiss.uni-passau.de> 1189 for never getting tired of assembling and testing new versions. 1190 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 1191 for providing ideas and code to improve FAST, XPK itself 1192 and doing various xBenchmarks on his A4000 and A3000. 1193 Ralph Schmidt <laire@uni-paderborn.de> 1194 for providing BAsm and BDebug [In my opinion the best 1195 development environment for assembler programs on the Amiga.] 1196 and doing some batch-xBenchmarks on his A4000. 1197 Michael van Elst <mlelstv@specklec.mpifr-bonn.mpg.de> 1198 for being so couraged to run one of the first alpha versions 1199 of the crawling mode on his A3000 during a large filetransfer 1200 --- and crash. 1201 Markus Illenseer <markus@TechFak.Uni-Bielefeld.DE> 1202 for enabling me the remote-use [and once -guru] of his A2000+68030 1203 and temporarily ripping all the 16Bit FAST RAM out for the sake 1204 of acurate xBenchmarks. 1205 Tobias Walter <walter@jazz.hall.sub.org> 1206 for letting me use his A1000 to test 11 totaly different and 1207 incompatible versions of FAST in one evening. 1208 @{"Matthias Meixner" LINK "Contact Matthias Meixner"} 1209 for doing some xBenchmarks when J�rg was `unavailable'. 1210 Markus Armbruster <armbru@pond.sub.org> 1211 for assisting me in the two weeks search for the 1212 _nonexistent_ timing-indeterministency-bug. 1213 1214Contact Addresses: 1215 1216 Ross Williams (ross@spam.ua.oz.au) 1217 1218 @{"Christian von Roques" LINK "Contact Christian von Roques"} 1219 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 1220@endnode 1221 1222@node FEAL 1223@toc SubLibs 1224FEAL is an XPK encryption sublibrary which implements the FEAL-N data 1225encryption algorithm in CBC1 mode. FEAL-N has been developed at the NTT 1226Communications and Informnation Processing Labs. in 1988. 1227 1228FEAL-N is a blockchifre, which encryptes a datablock of 64Bit to a 64Bit 1229codeblock using a 64Bit external key. FEAL mainly consists of a loop which 1230is taken N times. The loopbody encodes half of the data using a 16Bit 1231internal key and swaps the encoded half with the other one. The 64Bit 1232external key is expanded to N * 16Bit internal keys. 1233 1234FEAL was designed to be a replacement of DES. DES can be easy made fast 1235using special purpose hardware, but is a pain to be implemented in software 1236using conventional hardware. Since FEAL only uses 8Bit add, rol and eor 1237operations, it is designed to be implemented in software. 1238 1239Btw.: FEAL is one of the few algorithms which is easier to implement 1240using the 80x86 processorarchitecture than the 680x0 because of the 80x86s 1241splitable registers. 1242 1243 Safety 1244 ------ 1245 1246 Rounds Safety (? ;-) 1247 ------ ------------- 1248 4 unsafe, broken (Murphy 1990) 1249 1250 8 unbreakable for ``normal'' people 1251 1252 16 good Cryptoanalysists can decypher this with less 1253 (default) then testing all possible keys. But it can be valued 1254 as ``safe'' anyway. 1255 1256 32 There is no known better method of breaking this 1257 than testing all 2^64 possible keys. 1258 1259 64 Only paranoids will use this. 1260 ( But real paranoiac do not use FEAL ) 1261 1262 History of FEAL 1263 --------------- 1264 1265 1985 first proposal to ISO ( FEAL-1, FEAL-1', FEAL-2 ) 1266 1987 FEAL-4 presented on Eurocrypt. 1267 1987 attack on FEAL-4 by B. den Boer. ( Crypto 1987 ) 1268 => doubled the number of rounds: FEAL-8 1269 1988 FEAL-N proposed (N even >=4) 1270 1988 FEAL-NX proposed (N even >=4) 1271 different method to calculate partial keys 1272 => 128Bit key instead of 64Bit 1273 1274 1275 published attacks 1276 ----------------- 1277 1278 o B. den Boer (1987: FEAL-4; 100-10000 choosen plaintexts) 1279 o Murphy (1990: FEAL-4; 4 choosen plaintext) 1280 o Gilbert Chasse (1990: FEAL-8; statistically) 1281 o Bilham, Shamir (1990: FEAL-4. FEAL-8, FEAL-N, FEAL-NX) 1282 differencial Cryptoanalysis: 1283 => for up to 31 rounds better than testing all keys. 1284 o Gilbert (1991: FEAL-4, FEAL-6; 20000 knowm plaintexts) 1285 1286 published versions of FEAL 1287 -------------------------- 1288 1289 name rounds key internal key 1290 ---- ------ --- ------------ 1291 FEAL-1 4 64 4*16+2*32 1292 1293 FEAL-2 6 128 6*16+2*32 1294 1295 FEAL-1' 1296 FEAL-1.00 4 64 4*16+2*64 1297 FEAL-4 1298 1299 FEAL-2.00 1300 FEAL-8 8 64 8*16+2*64 1301 1302 FEAL-N N 64 N*16+2*64 1303 1304 FEAL-NX N 128 N*16+2*64 1305 1306 1307 Version History of xpkFEAL 1308 -------------------------- 1309 13101.0 First public release. 1311 13121.02 Fixed a stupid typo, which did not prevent the user from 1313 encrypting with an uneven number of rounds. 1314 13151.03 Previous versions filled the last block with junk, now 1316 the last encrypted byte is length&7. 1317 Minor speedups in the assembler part. 1318 1319 1320 Future Plans 1321 ------------ 1322 1323Support the other 3 standard modes. (ECB, CFB and OFB) 1324Improve the speed. 1325 1326 Contact Address 1327 --------------- 1328 1329 @{"Christian von Roques" LINK "Contact Christian von Roques"} 1330 1331 +----------------------------------------------------------+ 1332 | Questions regarding FEAL-N can be referred to: | 1333 | Mr. Shoji Miyaguchi | 1334 | Communications and Information Processing Labs., NTT | 1335 | 1-2356, Take, Yokosuka-shi, 238-03, JAPAN | 1336 +----------------------------------------------------------+ 1337@endnode 1338 1339@node HFMN 1340@toc SubLibs 1341This XPK sub library basically uses the same algorithm (dynamic huffman or 1342classic huffman) as found in the xpkHUFF.library. For more detailed 1343information about the huffman algorithm, take a look into HUFF.doc from 1344M.Zimmermann -- and skip the part that huffman compression & decompression 1345is pretty slow! In difference to HUFF, HFMN is FAST on compression and 1346decompression and produces a slightly better output. Although the basic 1347algorithm is the same, it is entirely different implemented, therefore 1348HFMN will not depack HUFF and HUFF not HFMN! 1349 1350 HFMN needs for private buffers (no xpkmaster.library buffers) 1351 1352 � 7.5 Kbyte packing memory 1353 � 5 KByte unpacking memory 1354 1355How does it work? 1356 1357 � First, I use heapsort to create the huffman tree, which is most 1358 responsible for packing speed. (heapsort is the second-best sort 1359 algorithm and is based upon binary trees) 1360 1361 � Second, (for decompression) I generate an (almost) optimal unpack code 1362 from the huffman tree. 1363 1364 � Third, I save the huffman tree recursivly. That is why I need max. 320 1365 byte to save a complete huffman tree. 1366 1367 020+ Version 1368 ------------ 1369I have experimented with 020+ code and rewrote the most used routines. My 1370huffman-code-translation-routine :) is reduced from 50 to 16 instructions, 1371and achieves a noticable speedup. (hmm, I like bitfield instructions.:-) 1372 1373.next move.b (a0)+,d2 ;incoming characters ( $00-$ff ) 1374 move.l (a2,d2.w*4),d3 ;huffman code 1375 move.b 3(a3,d2.w*4),d4 ;huffman codesize 1376 bfins d3,(a1){d5:d4} ;store huffman code 1377 add.l d4,d5 ;bitoffset + last codesize 1378 dbra d7,.next 1379 1380For decompression, the 020+ code produces no improvements. But there were 1381still some small optimizations possible, so decompression speed is improved 1382too. 1383 1384 Version History 1385 --------------- 1386 1387 V 1.16 - first public version. 1388 V 1.18 - 2 ways of decompression. 1389 V 1.19 - bugfix: uncompressable data returns now XPKERR_EXPANSION 1390 instead of XPKERR_SMALLOUTBUF. 1391 V 1.20 - V 1.27 - some experimental versions with 020+ code. 1392 V 1.28 - extra library with 020+ code for compression. 1393 - improved 000+ decompression code. 1394 V 1.29 - V 1.33 - some experimental version for the 1.34 bugfix. 1395 1396 V 1.34 - fixed a bug that i had added somewhere before 1.16. 1397 it should have caused problems only under bad circumstances, 1398 when the byte statistic was fibonacci like. 1399 (in fact, the decompression routine could not handle 1400 huffman codes longer than 16 bits, ups...) 1401 Thanks to Nicolas Pomarede for his superdetailed bugreport. 1402 (He analysed the code and told me exactly when and where it 1403 goes wrong :-) ) 1404 1405 V 1.35 - fixed a bug in the 020+ compression routine. 1406 (16 Bit overflow for number of bytes written to xsp_OutBuf 1407 was not handled correctly) 1408 Thanks to David Balazic for reporting this one. 1409 V 1.36 - 1.35 bugfix was not 100% ok. 1410 1411 1412 Contact Address 1413 --------------- 1414 1415 @{"Martin Hauner" LINK "Contact Martin Hauner"} 1416@endnode 1417 1418@node HUFF 1419@toc SubLibs 1420The idea of a huffman crunch is as follows: often used bytes (ie 8 bit 1421codes) get a shorter code than 8 bits, fi 5 bits. So everytime one of these 1422bytes occurs in the source file I save (in this example) 3 bits in the dest 1423file. To find out the optimum codes for each byte there is a simple method: 1424A tree is to be constructed so that the path from the root to each leaf 1425reflects the optimum (ie huffman) code. Unfortunately most computers (the 1426Amiga, too) are byte-oriented, which means a rather complex handling of 1427codes that are not a multiple of 8 bits. This results in pretty slow 1428compression & decompression. So this means that the xpkHUFF.library 1429probably will not be used for normal circumstances, but, as Dominik stated, 1430it may serve well as an example library. 1431 1432There are three different huffman crunch algorithms known: 1433 1434� static compression/decompression 1435� dynamic compression/decompression 1436� adaptive compression/decompression 1437 1438What are the differences? 1439 1440The static huffman uses a fix table to represent each byte of the source. 1441This, of course, makes sense only, if the structure of the data to be 1442crunched is known. In this case (for instance crunching an english text) a 1443fix table of codes is embedded in the code. Crunching other data than what 1444the table was generated for will probably result in worse compression or 1445even expansion. 1446 1447This is what a dynamic huffman is avoiding: it first creates a statistics 1448table reflecting the frequency every byte occurs with and generates a 1449special tree/table for this case, so the dynamic huffman does a good 1450compression for this special data. 1451 1452But there is something that can be improved, anyway: imagine, there is a 1453data block which contains many 'a's in its first part and many 'z's in the 1454last part.... The dynamic huffman would represent the 'a's and 'z's with 1455short codes, of course. But it probably would be even better if the 1456crunch/decrunch tree would reflect the *current* data beeing processed 1457instead of the whole block, thus in resulting shorter codes for 'a' and 'z', 1458depending of the position in the data block. This is what an adaptive 1459huffman deals with: it creates the tree while crunching, without doing any 1460statistics or tree creation before crunching. It permanently updates its 1461internal huffman tree. Therefore it does not have to include the information 1462about the decrunch tree in the crunched data. 1463 1464Final words about huffmans: 1465A stand-alone huffman will never achieve crunch results as fine as those 1466reached with most other crunchers, for these will not only regard the number 1467of occurances for each byte (as huffman does), but sequels of byte, too. 1468This means: If you create all permutations of a datablock, the huffman 1469crunched will always have the same length. Others will not, as they are 1470depending on the order of the crunched data, too. 1471 1472 Description 1473 ----------- 1474 1475The library 'xpkHUFF.library' implements a dynamic huffman crunch 1476algorithm, even though the adaptive might result in slightly better crunch 1477results. However, this is more complex to implement and I am using a maximum 1478buffer size of 64K, so this is a little bit like an adaptive huffman for 1479large files. 1480 1481If I should have lots of spare time I will probably implement an adaptive 1482huffman crunch algorithm. This new library will be called xpkHUFF, too, and 1483new xpkHUFF libraries will always handle output generated by earlier 1484versions. 1485 1486The xpkHUFF.library supports a totally unsafe (but a little bit better 1487than simple eor :-) encryption. Please note that crunch/decrunch speeds 1488decrease when encryption is used. 1489 1490 Implementation 1491 -------------- 1492 1493If you should see an errormessage saying output buffer too small while 1494crunching *and* encrypting, this means you tried to crunch and encrypt a 1495file that would crunched and encrypted be larger than the original file. 1496This should occur only with very small files (for I have a minimum file size 1497due to tables) or with files that have been crunched already and therefore 1498would expand during crunch. 1499 1500A technical note: this could also happen, if the last chunk of a file to 1501be crunched/encrypted would be dimensioned too small by xpkmaster.library. 1502 1503However, in this case you cannot encrypt the file. I know this could be 1504annoying and will think about a solution for this problem, but remember: 1505this encryption would not be safe, better if you used FEAL or IDEA for 1506secure encryption. 1507 1508 Last words ... 1509 -------------- 1510 1511I tried hard to debug this library with range checking while writing 1512bytes on crunching, and so on, but as in every code larger than, say 10 1513lines :-), there will be bugs. I do not know any bugs in this version, but 1514if you should meet one, please let me know via email. As usually, 1515reproducable bugs are preferred. Please add your configuration, programs 1516running (best if you try without startup-sequence!), and, most important of 1517all, add the file you tried to crunch! Thank you. 1518 1519 Version History 1520 --------------- 1521 1522; V 0.1 - 12-Jul-1992 : first version 1523; V x.yy - 18-Jul-1992 : first OK version 1524; V x.yy - 19-Jul-1992 : sped up decrunching 1525; V x.yy - 21-Jul-1992 : bug fixed in word/long decrunching: min pack 1526; chunk size now 3/5 1527; V x.yy - 21-Jul-1992 : replaced many subq/bxx with dbf (ie sped up 1528; crunching a little bit), bug fixed: there was 1529; a dbf counter wrong (one of my favorite 0/1 1530; problem bugs) 1531; V 0.50 - 29-Jul-1992 : added 68030+ cache optimized decrunch code 1532; V 0.51 - 01-Aug-1992 : byte decrunch improved, first code added, 1533; indicator byte for crunchmethod used added, 1534; 68030+ chache optimized code does not make 1535; sense any more, since byte decrunch fits to 1536; cache completely, now 1537; V 0.52 - 01-Aug-1992 : unsafe encryption supported 1538; V 0.53 - 03-Aug-1992 : slight improvements made to crunch code 1539; (+ 6K/s) 1540; V 0.54 - 03-Aug-1992 : inconsistence in expansion handling fixed 1541; V 0.55 - 03-Aug-1992 : bug fixed: expansion handling now considers 1542; table creation, too 1543; V 0.56 - 03-Aug-1992 : bug fixed: HUFF now can crunch files 1544; consisting of always the same byte (shame 1545; on me, termination criterium was wrong) 1546; V 0.57 - 03-Aug-1992 : Tree creation code partially rewritten 1547; V 0.58 - 05-Aug-1992 : bug fixed: wrong termination criterium for 1548; expansion check (my favorite 0/1 problem) 1549; V 0.59 - 06-Aug-1992 : now decrypting in a special buffer, not using 1550; InBuf (this is read only, I was told) any more 1551; V 0.60 - 07-Aug-1992 : added extra memory required during 1552; packing/unpacking 1553; V 0.61 - 08-Aug-1992 : expansion check changed, renamed from 1554; xpkDHUF.library to xpkHUFF.library thus 1555; corresponding to the possibility of handling 1556; adaptive huffman codes later without having 1557; an inconsistence in the name 1558; V 0.62 - 10-Aug-1992 : Flag XPKIF_MODES removed (I do not have modes 1559; yet (but I have a mapping code :-=)) 1560 1561Contact Address: @{"Marc Zimmermann" LINK "Contact Marc Zimmermann"} 1562@endnode 1563 1564@node IDEA 1565@toc SubLibs 1566 Patent 1567 ------ 1568 IDEA is registered as the international patent WO 91/18459 1569 "Device for Converting a Digital Block and the Use thereof". 1570 For commercial use of IDEA, one should contact 1571 1572 ASCOM TECH AG 1573 Freiburgstrasse 370 1574 CH-3018 Bern, Switzerland 1575 1576 Description 1577 ----------- 1578 1579IDEA is an XPK packer sublibrary which implements a highly optimized form 1580of the IDEA encryption algorithm. 1581 1582IDEA (International Data Encryption Algorithm) is a block cipher 1583developed by Xuejia Lai and Prof. Dr. J. L. Massey at the Swiss Federal 1584Institute of Technology. See patent for information on any commercial 1585use of this algorithm. Especially, this library is not only claimed by 1586the copyright of the author and the copyright of the author of the used 1587IDEA kernal routine, but by the copyright of the IDEA originators and 1588their patent, too. 1589 1590This implementation of the algorithm was done by Andr� Beck, Dept. of 1591Computer Science, Technical University of Dresden, Germany. 1592 1593xpkIDEA.library gives a chunk based access to the most common encryption 1594methods, using the IDEA cipher as the encryption function. The IDEA cipher 1595is known to be somewhat slow. It performs 34 multiplications modulo 2^16+1 1596for every 64 bit data packet, so it must have limited performance on a 1597plain 68000 processor. This library uses the heavily hand optimized, 1598permuted, macrotized and partially unrolled 68000 assembler implementation 1599of IDEA by Colin Plumb. Therefore, the kernal IDEA routine and it is 1600macros are copyright by Colin Plumb. 1601 1602In difference to the most wide spread compressors distributed with XPK, one 1603should know something about IDEA before using it. First, IDEA is completely 1604no compressor, it only encrypts or decrypts data. A password must be 1605specified with first calls to "pack" or "unpack" a chunk. Furthermore, the 1606password given on encryption MUST restore the original chunk contents, 1607otherwise the password will be treated as incorrect. This is a tribute 1608to the XPK architecture and its safety, but has the disadvantage of 1609preventing you from doing things like a triple crypt, what means to first 1610encrypt a chunk with password 1, then decrypting it with password 2 and 1611last encrypting it with password 3, all three passwords different. 1612 1613 Encryption Methods 1614 ------------------ 1615 1616IDEA is a cipher used for encryption in this library, what means it is a 1617function taking one data block and an encryption key as input and 1618producing one data block as output. The purpose of this function is to 1619generate a very random result from normaly highly redundant input, in other 1620words to make White Noise of bits from a regular, low entropic bit stream. 1621The IDEA data blocks are sized 64 bits, where the key has 128 bits in its 1622unexpanded form (DES has a key of 56 bits). One now may use this function 1623in different ways. The simplest encryption is to take the input chunk block 1624by block, driving it through the IDEA function, and building the output 1625chunk from the result. This mode is called Electronic Code Book (ECB). 1626But an Code Book based encryption is not the state of the art, because it 1627is somewhat easy to crack (even ECB using IDEA is not easy to crack, only 1628a bit easier than the following modes). One can imagine, that including the 1629chunks contents (which is to be crypted) itself into the encryption will be 1630much safer. Consider a simple ECB to encrypt text, generated by the function 1631 1632 out_character = (in_character + 1) MODULO num_of_characters. 1633 1634This is nothing other than incrementing every character, f.i. making A to B, 1635F to G and Z back to A. So the word FOOBAR will be crypted to GPPCBS, and 1636nobody will see what it is on the first visit. But there are also people 1637called Cryptologists, and cracking such codes is their job. Simple methods 1638of cracking are especially based on the probability of characters in 1639different languages. They know e is a very often found letter in indo 1640european languages, and if they find one character very often in the 1641crypted text, this one may be an e. If it is sure that it is an e, one can 1642insert it in the complete crypted text where the cracked character was. 1643 1644The method to prevent such simple cracks is based on chaining the produced 1645output back into the crypt function with some delay. 1646Consider 1647 1648 out_character = ((in_character + last_out_char)+1) MODULO num_characters 1649 1650with an initial last out character of 'C'. 1651 1652FOOBAR gets JAQTVL using this code and nobody can see that an double O was 1653in the input. So it is more complicated to crack messages crypted with this 1654code, because one MUST start at the beginning of the text. It is also 1655possible to increase the number of ,,states of remember'' we are using, 1656for instance by not using the last_out_char but the seventh_last_out_char 1657and using 7 different initial values for them. 1658 1659The method used above is very similar to a common encryption method called 1660Cipher Block Chaining (CBC) with one state of remember (CBC 1). 1661 1662The difference to ECB in schematic view: 1663 1664ECB electronic code book mode 1665 y[i] = IDEA(z, x[i]) 1666 x[i] = IIDEA(z, y[i]) 1667 1668CBC cipher block chaining mode 1669 y[i] = IDEA(z, x[i] ^ y[i-N]) 1670 x[i] = IIDEA(z, y[i]) ^ y[i-N] 1671 1672with 1673 1674 x[i] is the input block number i 1675 y[i] is the output block number i 1676 z is the encryption key 1677 N is the number of states of remember (at least one) 1678 IDEA is the encryption function using the IDEA cipher 1679 IIDEA is the corresponding decryption function 1680 ^ means the XOR of the operands (Bitwise Exclusive Or) 1681 1682There are two additional modes often used with encryption. See the following 1683schematics: 1684 1685CFB ciphertext feedback mode 1686 y[i] = x[i] ^ IDEA(z, y[i-N]) 1687 x[i] = y[i] ^ IDEA(z, y[i-N]) 1688 1689OFB output feedback mode 1690 h[i] = IDEA(z, h[i-N]) 1691 y[i] = x[i] ^ h[i] 1692 x[i] = y[i] ^ h[i] 1693 1694As you see, all the chaining modes have additional parameters determining the 1695result of the crypt. Not only the key determines the resulting chunk for a 1696special input chunk, but also the number of states of remember used by the 1697mode and the values used to initialize the states (in CBC 1 coding block 1698#0, you need block #[i-1], but you have no block #-1, so you have to give 1699some initial value for it). For CBC 8 you have to give 8 initailizers, 1700and so on. 1701 1702The xpkIDEA implementation uses the following XPK modes for different 1703encryption methods: 1704 1705 0.. 25 ECB 170626.. 50 CFB 170751.. 75 OFB 170876..100 CBC 1709 1710As you see, the modes were ordered to somehow match the scheme given by the 1711most XPK packers, with 0..100 mapped to increasing efficiency and decreasing 1712speed. There are neither big differences in speed nor in efficiency of the 1713used modes, and the mapping used is easy to remember. Especially one gets 1714very simple from the mode used to the encr. method and state number by 1715subsequent subtractions of 25 from the mode: 1716 1717 IDEA.79 -> 79 - (3*25) = 4 , so mode 3 (CBC) with 4 states applies. 1718 1719The default method used when no mode is explicitely given is CBC1, 1720i.e. the mode IDEA.76 1721 1722The presented speed (in KByte/second) is not very exact. This is mainly 1723caused by the varying cycle count of the 68000's mul instruction. The 1724encryption will be faster with the all-zero-key and slower with the 1725all-ones-key. Try around with key values 1726 #0 1727 #5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a 1728 #ffffffffffffffffffffffffffffffff 1729to see the differences. 1730 1731Not only IDEA.100 is a very safe encryption, also IDEA.75 and IDEA.50 may 1732be good for safe results. They are modes with 25 states, so one may give 173325 (!) different initializers to the password, which must all be known to 1734get this decrypted again. The code is developed in a way that no speed 1735loss will occure even using much states. At the other hand, a open connection 1736with this sublibrary for packing or unpacking forces the allocation of around 1737600 bytes of memory. If you are low on memory, the library may return a 1738matching error condition. 1739 1740 The Password 1741 ------------ 1742 1743You may ask now, how to give different initializers to the encryption 1744modes which use them ? Therefore, the password parsing routine within this 1745library is more complicated than normal ones. An IDEA password consists 1746of the key value and a optional set of initializers, both specified either 1747as a plain ascii string to be hashed or as the explicite hexadecimal value. 1748 1749The syntax is as follows: 1750 1751 1752 <password> ::= <keyspec>[<initializer>]*. 1753 <keyspec> ::= <valuespec>. 1754 <initializer> ::= ":"<valuespec>. 1755 <valuespec> ::= [<charstring>|<hexstring>]. 1756 <charstring> ::= ["!".."~"]*. 1757 <hexstring> ::= "#"["0".."9"|"a".."f"|"A".."F"]*. 1758 1759 so possible passwords are f.i.: 1760 password 1761 password:heut:ist:montag 1762 #738494ad53ae2c1b736218ac12abaacc:nix:hexa:oder:doch:#4455663311223311 1763 : <-- this results in the all-zero-key. 1764 passwd::::ini4 <-- initializers 1..3 are zero 1765 1766 Its useless to specify any initializers with ECB 1767 Its useless to specify more then N initializers for mode [CBC|CFB|OFB] N 1768 The maximum number of initializers is 25 1769 charstrings may have any number of characters 1770 hexvalues for keyspec have to fit in an OCTAWORD. (16 Byte) 1771 hexvalues for initializers must fit in a QUADWORD. (8 Byte) 1772 unspecified values (key/initializers) are zero. 1773 1774 1775If you do not initialize a value, it will be zero. Any syntactic or 1776semantic error in the password specification will raise the error 1777XPKERR_WRONGPW. The '#' character is used to introduce hex values because 1778many shells would missinterpret $ even if it appears in doublequotes. 1779The hash routine currently used in this password parser is not very strong. 1780String passwords should be at least 12 characters long to give a nice 1781key. 1782 1783 Technical Info 1784 -------------- 1785This lib is completely written in assembler using a68k and the 1.3 includes. 1786It was developed within around 10 hours of work distributed over more than 178714 days (better to say nights). 1788The author could only test it on an 1 Meg chip no fast 7.14MHz 68000 A500 1789under Kickstart 1.3. The source is now around 30000 bytes and may contain 1790some bogus. If you find any bugs report them to me via the email address 1791given below. 1792Make sure the output buffer is at least the size of the input buffer plus 1793XPK_MARGIN, even if this is on decompression more then the original chunk 1794size. This library relies on this behavior, which is correctly done by 1795xpkmaster. 1796 1797As already stated in the section Disclaimer, the author gives no warranties 1798for the proper function of this software. Additional, he cannot give any 1799guarantee that IDEA itself is a useful encryption standard. It SEEMS to be 1800very strong, but it is still under analyzation by some organizations like 1801the NSA and similars. If you are interested in the theoretics of this 1802algorithm, ask me for some hints. 1803 1804Contact Address: 1805See section Patent for information on how to reach the authors of the IDEA 1806cipher. 1807 1808If you want to get in contact with the author of the fast idea routine used 1809within this library, contact Mr. Colin Plumb at: colin@eecg.toronto.edu 1810@endnode 1811 1812@node IMPL 1813@toc SubLibs 1814This XPK sub-library uses basically the same algorithm as found in the 1815Imploder, but without the specifics needed for compressing self-contained 1816executables. 1817 1818A quote from the Imploder 4.0 technical manual says it all :-) 1819 1820IMPL does LZ77 like compression with a, per mode, static Huffman like 1821coding step on the various parts of the skip, offset and length tuples. 1822Due to the efficient encoding, a tuple can require less than 12 bits, and 1823thus strings of 2 bytes length and up are encodable with a decent gain 1824(given small Huffman patterns corresponding to likely circumstances). 1825 1826The default compression mode is 100 which means that the actual compression 1827mode used depends on the chunksize. The default chunksize is 64K. In 1828general, this mode produces the best compression ratio, although the mode 1829range 76..98 (1.00*max) will sometimes produce better results. 1830 1831The current version of xpkIMPL.library will, by default, react to a BREAK 1832signal (CTRL-C) while compressing. Compressing a chunk (especially on 1833unaccelerated Amiga's) can take quite a bit time, so allowing the user to 1834break-off compression is useful. For now, it is not possible to turn this 1835feature off! 1836 1837Version History: 1838 18390.01 Well known Imploder compression algorithm now included in a XPK 1840 sublibrary. 18410.19 Improved robustness. 18421.00 Much needed documentation added. 1843 1844Contact Address: @{"Peter Struijk" LINK "Contact Peter Struijk"} 1845@endnode 1846 1847@node MASH 1848@toc SubLibs 1849xpkMASH is an XPK compression sublibrary whose main purpose is to 1850decrunch fast and have an excellent crunch factor. The sublib is using 1851LZ77 compression and a special method to write matches... MASH now 1852normally uses 256KB for its tables, but reduces the size of the 1853hashtable if memory is scarce. (it could crunch even with 64KB+4KB) 1854Compressing with a small hashtable naturally is very very slow. 1855Default chunk size is 64KB. The compressor uses lazy match evaluation 1856which slowed it down quite a bit. 1857 1858This sublibrary has several modes: 1859 1860 Mode Strings to be searched 1861 ------ ------------------------ 1862 0-09 1 ;high speed - but low CF 1863 10-19 2 1864 20-29 4 1865 30-39 8 ;good for most executables 1866 40-49 16 1867 50-59 32 1868 60-69 64 1869 70-79 128 ;this should be used for text files 1870 80-89 256 1871 90-99 512 1872 100 1024 ;the best, the slowest 1873 1874The second column is showing, how many matches should be compared 1875- the more searched strings - the better results you will get. 1876formula is simple 2^(MODE/10). 1877MODE 70 is now runing as fast as NUKE on my A1200 1878and if you want to use some higher modes - you will get result better 1879for only a few bytes, but slowdown will be very noticeable. 1880(But for crunching I am always using the best mode anyway :-)) 1881 1882 !!! The source for this version is not released !!! 1883if you want to see it anyway, drop me an e-mail and I will send you it. 1884 1885I still want to do some improvements. Probably even change format of stored 1886data to reach better decrunch speed and possibly use some more MC68020 1887instructions in this case. You do not have to worry, this library will also 1888decrunch old format. Send me an e-mail what you would like to see in newer 1889version of this library. But this newer format will always need 1890256KB of memory so it could be a problem for some people. 1891If you think this library is worth some money, you could send them 1892It will speed up development :-) 1893 1894 "Thank you"s must go to: 1895 ------------------------ 1896@{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 1897 for XPK standart. (Try to respond to my e-mails sometimes :-)) 1898 1899@{"Christian von Roques" LINK "Contact Christian von Roques"} 1900 for correcting some parts of this document file, 1901 and also for releasing his source, so I could use some parts 1902 of it in my library (XPK interface). 1903 1904@{"Karsten Dagef�rde" LINK "Contact Karsten Dagef�rde"} 1905 for making benchmarks and other cooperation 1906 1907more people should be in this list - authors of Zip, Lha, Arj, ... 1908but I would have to make some deep research for them. 1909 1910 History 19110.5 Many errors, the biggest problem was bad writing of bits string. 19120.7 Most errors have been debuged 19130.8 Last byte has not been saved 19140.9 On the first look, normaly working version of the sublibrary with 1915 fixed hash table - size 64KB 19161.0 The big improvement in memory allocating; 1917 memory is allocated before each chunk compression and deallocated 1918 after this chunk is compressed (usefull if you have installed 1919 statram.device) 19201.01 Hash size was increased from 64KB to 128KB (16 bits) 19211.05 Hash is allocated dynamicaly - when is large memory free - large hash 1922 is used. Starting with 128KB, 64KB, 32KB, .... ,512 bytes 19231.15 Seems to work perfectly for me 1924 1925First public release: 19261.16 I suppose last bug has been removed - value of register D4 1927 was not saved on return. Also most long word instruction have 1928 been rewritten to word oriented instructions (useful for MC68000) 19291.26 Several speed up improvements - decompression goes about 50 kB faster 1930 1931Unreleased 19321.30 New crunch mode - uses 256KB of memory for its buffers 19331.40 Removed checking of two bytes in match 1934 it is not needed when two-byte hash is used 19351.53 Removed zero length write when chunk is uncrunchable 1936 Diavolo is a little bit odd and uses this value for DIVS 1937 even when its not valid -> caused GURU 19381.61 Removed bsr call from scanning routine. 1939 1940Released 19411.77 Prepared for release - there are still many things to improve, 1942 but it already has a very good speed. So I am releasing this 1943 version. 19441.98 Well many new checks have been added to prevent a to deep scan 1945 when better match cannot be found. Even a little bit better 1946 alghorithm was used for lazy_eval -> better CF. 1947 1948Contact Address: @{"Zdenek Kabelac" LINK "Contact Zdenek Kabelac"} 1949@endnode 1950 1951@node NONE 1952@toc SubLibs 1953NONE is only a dummy packer, which was an programming example in the 1954first distribution. It only copies the data to the resulting file. 1955(With 52 or more bytes header) 1956 1957It may be useful with xpkarchive.library, because it gives the option of 1958no crunching like in LhA. 1959@endnode 1960 1961@node NUKE 1962@toc SubLibs 1963NUKE is an XPK packer sublibrary which implements a highly optimized form 1964of the popular LZ77 compression algorithm. This is essentially the same 1965algorithm used in PowerPacker, Imploder and (among other algorithms) in 1966the LZH/LHA packers. 1967 1968Most applications of packers mean packing once and unpacking many times. 1969One example is a PD program that gets distributed around the world, or a 1970compressed program on the hard disk the needs to be decompressed when 1971loaded. NUKE tries to�be fast at decompression, thus restricts itself from 1972applying fancy algorithms (Huffman, Ari-coding). In order to achieve 1973reasonable compression factors anyway, it scans a very long range (more 1974than 24K) for identical byte sequences and if it finds any, it outputs 1975offset and length instead of the bytes themselves. 1976 1977Of course scanning such a long range for duplicates is quite a CPU 1978intensive process. I have tried to make it as fast as possible, and with 1979around 35K/sec (A3000) I would say I have come close to the best that can be 1980done with this approach. There is a drawback, though. The compression must 1981use large hashing tables to reach this speed. I have made sure that NUKE is 1982still usable on a plain 512K Amiga, but you will not be able to run many 1983things besides NUKE while you are packing. There is, by the way, no increase 1984in mem needs with increasing file size. 1985 1986Version History: 19871.0 First public release. 19881.2 Does compress slower, but a bit better. 1989 Decompression is faster than V1.0. 19901.3 Fixed excessively long 2 byte matches [there were files, on which 1991 NUKE was not bijective!] 19921.4 added new startup header 19931.5 recompiled to fix error with DiskExpander 19941.6 better library start header 19951.7 fixed low-mem error 1996 1997Contact Addresses: @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"}, @{"Christian von Roques" LINK "Contact Christian von Roques"} 1998@endnode 1999 2000@node RAKE 2001@toc SubLibs 2002RAKE is an XPK packer sublibrary which implements a highly optimized form 2003of the popular LZ77 compression algorithm. It uses static huffman coding 2004for the 'len' and a three-step coding for the 'offset' information. The 2005major feature of this packer is the highly optimized algorithm for tracking 2006down redundant data. 2007 2008RAKE supports four modes at compression (see below). 2009 - Scanner & Coder together fit in 68020 instruction cache 2010 - Hashbuffer-size will be reduced downto 0.5Kb, if memory is short 2011 2012 History 2013 ------- 2014 2015Sep-9-94 V1.0 First public release (68000,68020) 2016 [...] 2017Jul-1-95 V1.6 - Scanning algorithm improved. 2018Sep-6-95 V1.7 - 68020 version now checks if there is an appropriate 2019 processor (68020 or better) 2020 2021Contact Addresses: @{"Karsten Dagef�rde" LINK "Contact Karsten Dagef�rde"} 2022@endnode 2023 2024@node SHRI 2025@toc SubLibs 2026SHRI is an XPK packer sublibrary which implements a compressor, highly 2027optimized for compression rate. It uses offset/len encoding with 2028adaptive arithmetic aftercoding for best compression results. Its 2029compression rate is better than that of most other packers, e.g. lha, 2030zoo or powerpacker. 2031 2032 It supports 7 modes, which differ in the size of the dictionary: 2033 2034 Modes Dictionarysize 2035 ------ -------------- 2036 0- 14 1k 2037 15- 28 2k 2038 29- 42 4k 2039 43- 56 8k 2040 57- 70 16k 2041 71- 84 32k 2042 85-100 64k 2043 2044A larger dictionarysize gives a higher compression rate and faster 2045decompression, but slows down compression. 2046 2047This library is not meant to be used for online-compression as it is used 2048in e.g. XFH. It is meant to be used for achieving highest compression-rates. 2049These can be useful, when transferring files via modem, for a backup to a 2050slow medium (floppy disks) when you have a fast computer or for creating 2051archives with the xpkarchive.library. 2052 2053The decompressionspeed of this version is about 82% higher than that of 2054version 1.0. The compression is about 5% faster. 2055 2056 History 2057 ------- 2058V0.2 First public Release 2059V0.3 XpksPackChunk retured XPKERR_CORRUPT instead of XPKERR_EXPANSION 2060 - Bug fixed in V0.3 2061V0.4 SHRI could not decompress files, which where partly compressable 2062 - Bug fixed in V0.4 2063V1.0 New Version with improved compression- and decompressionspeed 2064V2.1 Improved decompressionspeed, all relevant parts of the decompressor 2065 are now written in assembler. 2066V2.2 Removed important bug in the decompression-part, that produced errors 2067 with chunksizes above 32767 bytes 2068 2069Contact Address: @{"Matthias Meixner" LINK "Contact Matthias Meixner"} 2070@endnode 2071 2072@node SMPL 2073@toc SubLibs 2074SMPL is a XPK sublibrary implementing dynamic huffman coding over 2075variations of datastream. If that sound too complicated, I suggest you 2076read docs for @{"DLTA" LINK DLTA} and @{"HUFF" LINK HUFF}, in that order. In fact, DLTA was made to be 2077used as preprocessor for other XPK packers. 2078 2079Then why did I code SMPL? Think this: how many music programs you know that 2080support XPK? Yes, I know I can always use XFH so I can pack all my 2081data, but if I have first fed data thru DLTA and then another compressor, 2082then XFH only decompresses the latter. So I still need XPK supporting 2083program to pack my samples efficiently. 2084 2085SMPL overcomes this by including @{"DLTA" LINK DLTA} coding into same library. I chose 2086to use huffman coding for actual packing as it seemed to give best average 2087compression. I snatched the huffman code from @{"xpkHUFF.library" LINK HUFF} and 2088tweaked it a bit for faster (de)compression. 2089 2090Some samples were packed better with simple @{"HUFF" LINK HUFF} without delta precoding. 2091If I find a way to determine output size from frequency table (ie. without 2092building huffman tree) I will add non-delta packing to SMPL. 2093 2094I tested @{"DLTA" LINK DLTA}+SMPL mainly to see if there would be any use for recursive 2095delta, but less than 100K of all data packed marginally better when fed 2096thru double delta. 2097 2098Three percent difference between SMPL and @{"DLTA" LINK DLTA}+@{"HUFF" LINK HUFF} comes from two 2099things: 2100 1) xpkmaster.library adds some bytes to DLTA coded files 2101 2) I store huffman tree in more compact way 2102 2103Version History: 1.00 First public release. 2104 2105Contact Address: @{"Jorma Oksanen" LINK "Contact Jorma Oksanen"} 2106@endnode 2107 2108@node SQSH 2109@toc SubLibs 2110SQSH is an XPK packer sublibrary which implements an optimized LZ based 2111algorithm combined with a 8 bit delta compression algorithm. 2112 2113This packer was especially made for packing 8 bit Samples and ProTracker 2114style modules. It is NOT a lossy compression library, so NO quality-loss 2115will occur when packing Samples with this library. 2116 2117SQSH is pretty fast at decompression (300K/s on A3000) so is very well 2118suited to compress Modules and Samples since these will typically be packed 2119once and unpacked many times. It is slow at compression (25K/s on A3000) 2120mainly because every part of the file has to be checked twice to see what 2121the better compression method would be. 2122 2123In order to achieve reasonable compression of other types of files 2124(Executables, Textfiles) this packer will scan a long range (about 20K) for 2125identical byte sequences and if it finds any, it outputs offset and length 2126instead of the bytes themselves. Scanning such a long range for duplicates 2127is a CPU-intensive process. I have tried to make it as fast as possible 2128(about 25K/s on A3000) but @{"NUKE" LINK NUKE} proves it can be done faster :-) 2129 2130In the archive also is included a 68000 version of this library. Sorry 2131all you 68000 users for the long delay, but I only received one message 2132asking me for a 68000 version. It was Edmund Vermeulen who eventually 2133 2134Contact Address: @{"John Hendrikx" LINK "Contact John Hendrikx"} 2135@endnode 2136 2137@node C-Utils 2138 @{"xBench" LINK xBench} benchmark tool for XPK sub libraries 2139 @{"xDir" LINK xDir} directory lister 2140 @{"xLoadSeg" LINK xLoadSeg} LoadSeg function patch 2141 @{"xPack" LINK xPack} XPK packer and unpacker for OS2.0 2142 @{"xPK" LINK xPK} XPK packer and unpacker for OS1.3 2143 @{"xQuery" LINK xQuery} XPK sub library information 2144 @{"xScan" LINK xScan} scanner command for XFH 2145 @{"xType" LINK xType} XPK type command 2146 @{"xUp" LINK xUp} XPK unpacker 2147 2148 @{"XpkMasterPrefs" LINK "Preferences - XpkMasterPrefs"} Preferences control 2149 2150 Program histories are in source code. 2151@endnode 2152 2153@node xBench 2154@toc C-Utils 2155SYNOPSIS 2156 xBench FILENAME/A,PASSWORD/K,METHOD/M,TEST/S,ALL/S,SAVE/K, 2157 LOG=LOGFILE/K 2158 2159 FILENAME Is needed always. This is the name of the test file(s). 2160 You can specify names containing normal dos patterns. 2161 PASSWORD When given, xBench tries all crunching methods with and 2162 without password (when possible). 2163 METHOD Here you may specify crunchers (multiple when needed). The 2164 format is <packername>[.<mode>] When no METHOD keyword is 2165 used, all packer libraries are scanned. 2166 Examples: NUKE, MASH.20, RAKE.100, MASH NUKE SHRI.100 2167 TEST When you give this option, the decrunched buffer is 2168 checked against the source file, errors are reported. 2169 ALL This option scans all modes, when no special one is passed. 2170 >>>> Use this with care, as there are 101 modes! <<<< 2171 SAVE Here you may specify a directory, where all packed data is 2172 saved to. When nothing is specified, the data will not be 2173 saved. When a buffer error is reported, the saved data of 2174 following modes may be false, as some crazy sub libraries 2175 destroy source buffer. 2176 LOGFILE Additionally to normal output you may create a logfile to 2177 store the results. It covers same informations as printed 2178 on screen. 2179 2180 When no METHOD keyword is passed, all possible types are used. 2181 2182 NOTE: When no password is given, all libraries needing passwords 2183 are skipped. When password is given all libraries are called 2184 with and without password. The real output depends on the fact, if 2185 the library allows, needs or does not support password. 2186 2187 When no mode information is passed for METHOD keyword (f.e. NUKE) 2188 or with no METHOD keyword, all modes are scanned. For every type 2189 only the highest mode number is used. In case you specify ALL 2190 keyword, all modes from 0 to 100 are used. 2191 2192DESCRIPTION 2193 With xBench you can create benchmarks for XPK sub libraries. The 2194 benchmarks created will be relative to the system (hardware and 2195 possibly software) it is run on and to the file used as data. To 2196 generate standard XPK benchmarks shown in information data of the 2197 sub libraries see end xpksub.doc. There standard environment and 2198 standard files are described. 2199 2200 xBench uses timer.device for as accurate measurements as possible, 2201 but the given time results can differ from call to call. 2202 2203 NOTE: Benchmarks are different for different file-types, so you 2204 may produce benchmarks for text, exe-files, sounds, ... 2205 2206NOTES 2207 Best method to use is : xBench <filename> TEST PASSWORD TestPwd 2208 2209 If error messages "FileSize false ..." or "Decrunched buffer ..." 2210 occur, the library producing these errors should be deleted and 2211 the library author should be informed of the error. 2212 2213 When the program crashes, this is most time a problem of a 2214 sub library. You can find the bad library when looking into 2215 LIBS:compressors. The bad library is most time the one after the 2216 library that produced last output (when files are sorted 2217 alphabetically). 2218 Reboot your system and retest the library with xBench's METHOD 2219 option. If it crashes again, delete it! 2220 2221WARNING 2222 For accurate measurements xBench have to Forbid() task rescheduling 2223 while packing and unpacking. This means that multitasking will be 2224 disabled while xBench is running. Note that doing a Forbid() for a 2225 long time is potentially dangerous. 2226 2227THE OUTPUT 2228 2229 The output data shows as following: 2230 2231File 'C:Ed' with a size of 25028 bytes. 2232Type Num Version P CSize CTime CSpd UTime USpd Rate 2233BLZW: 14 3.2 22220 0.02 1251400 0.01 2502800 11.3 2234 2235 Type: the library type 2236 Num: the crunch mode which was used 2237 Version: version and revision of the library 2238 P: when password was used a "*" is shown here 2239 CSize: file size after crunching 2240 CTime: time used to compress the file (seconds) 2241 CSpd: compression speed (bytes/sec) 2242 UTime: time used to uncompress the file (seconds) 2243 USpd: uncompression speed (bytes/sec) 2244 Rate: crunch factor 2245 2246COPYRIGHT 2247 Freely distributable for noncommercial use. 2248 2249AUTHOR 2250 @{"Dirk St�cker" LINK "Contact Dirk St�cker"} 2251@endnode 2252 2253@node xDir 2254@toc C-Utils 2255SYNOPSIS 2256 xDir 2257 xDir [filenames] 2258 xDir [directories] 2259 2260DESCRIPTION 2261 xDir shows a listing of all files in the current directory (first 2262 form), or of a number of files or directories. Wild cards are not 2263 recognized. Files are sorted alphabetically. 2264 2265 xDir sums up the uncompressed and compressed total sizes of all 2266 files shown. 2267 2268COPYRIGHT 2269 Freely distributable for noncommercial use. 2270 2271AUTHOR 2272 @{"Dirk St�cker" LINK "Contact Dirk St�cker"} 2273@endnode 2274 2275@node xLoadSeg 2276@toc C-Utils 2277OVERVIEW 2278 xLoadSeg wedges into LoadSeg() and NewLoadSeg (if available) and 2279 allows to directly run programs that were compressed using the XPK 2280 standard. They are decompressed while being loaded. xLoadSeg uses 2281 less than 700 bytes when installed. Should not really bother you. 2282 2283HOW TO USE 2284 Just start 'xLoadSeg' from your startup-sequence. No need to 'run' 2285 it. If you want to remove it, use 'xLoadSeg -q'. The 700 bytes will 2286 be lost. Do not try to start from Workbench. 2287 2288HISTORY 2289 2290 1.0 First release, based on PPLoadSeg by Nico Fran�ois 2291 1.1 Complete rewrite: Now able to remove patch 2292 Symbol and Debug Hunks are now skipped, Overlayed Files 2293 gracefully fail instead of crashing the machine. 2294 2295COPYRIGHT 2296 This program is Public Domain. You may freely use it, spread it, 2297 enhance it or even delete it. No warranties either expressed or 2298 implied, use it at your own risk! 2299 2300AUTHOR 2301 @{"Christian Schneider" LINK "Contact Christian Schneider"} 2302@endnode 2303 2304@node xPack 2305@toc C-Utils 2306SYNOPSIS 2307 xPack FILE/M/A,METHOD/K,MINSIZE/N/K,SUFFIX/K,PASSWORD/K, 2308 ALL/S,FORCE/S,PROGRAM/S,XSCAN/S,LOSSY/S,QUIET/S 2309 2310DESCRIPTION 2311 xPack is a command line interface to the XPK compression library. 2312 It was made to enable you to pack (or unpack) many files quickly 2313 and comfortably, exspecially for use with the XFH-Handler. 2314 xPack needs OS 2.04 or newer. 2315 2316 Main features: 2317 - supports patterns 2318 - can scan complete directory trees 2319 - protection flags, filenote and date of the files are NOT changed 2320 - packed files will not be repacked by default 2321 2322 For more details about XPK read the documentation supplied. 2323 2324ARGUMENTS 2325 2326FILE You can supply as many files, directories or patterns 2327 as you want. 2328METHOD the compression algorithm to be used 2329MINSIZE All files which are smaller than this value (in bytes) 2330 will not be crunched (default 512 bytes). 2331SUFFIX add/remove supplied suffix if packing/unpacking 2332PASSWORD optional Password for encryption (or decryption) 2333ALL scan through directory trees 2334FORCE Files will be packed even if they are already XPK 2335 packed and/or their size increases. 2336PROGRAM pack only executables (e.g. for @{"xLoadSeg" LINK xLoadSeg}) 2337XSCAN create filenotes for fast directory access with XFH 2338 (like @{"xScan" LINK xScan}) 2339LOSSY permit lossy packing 2340QUIET No progress report is printed while packing. 2341 2342 Examples: 2343 2344 xPack SYS:MetaFont METHOD @{"NUKE" LINK NUKE} ALL 2345 2346 or 2347 2348 xPack Docs/#?.doc METHOD @{"IMPL" LINK IMPL}.40 SUFFIX .xpk MINSIZE 1024 2349 2350 or 2351 2352 xPack Secret.txt METHOD ENCO PASSWORD Joshua 2353 2354 or (Decrunch) 2355 2356 xPack Archive/#?.xpk Archive/#?.pp QUIET 2357 2358HISTORY 2359 1.0 (XPKSmart) first internal Release 2360 1.0 (xSmart) 2361 program renamed on a request by @{"Urban 'XPK' M�ller" LINK "Contact Urban Dominik M�ller"} 2362 progress display fits better in the CLI window now 2363 check for increase of size by packing with XPK implemented 2364 1.0 program renamed again on a request by @{"Urban 'XPK' M�ller" LINK "Contact Urban Dominik M�ller"} 2365 "FORCE", "PASSWORD" and "SUFFIX" argument implemented 2366 file handling changed, slower but more secure 2367 removed Enforcer hit 2368 1.1 no problems with WShell anymore 2369 if xPack is started with OS 1.3 a message is printed 2370 instead of displaying a recoverable Alert 2371 1.2 added "PROGRAM" parameter 2372 suffixes may be removed while unpacking 2373 1.3 wasted my time with a special function for ILBM-files 2374 added "QUIET" parameter 2375 "FORCE" is on automatically if a password is supplied. 2376 1.4 changed "FILE/M" to "FILE/M/A" in template 2377 added "XSCAN" and "LOSSY" parameter 2378 1.5 "LOSSY" always active in 1.4 2379 2380COPYRIGHT 2381 xPack is free to be spread on public-domain and shareware disks as 2382 long as they are sold for a reasonable charge that is less than $6. 2383 This applies not to Fred Fish, he and ONLY he can take more money. 2384 For use in commercial products the permission of the author is 2385 required. 2386 2387AUTHOR 2388 @{"Matthias Scheler" LINK "Contact Matthias Scheler"} 2389@endnode 2390 2391@node xPK 2392@toc C-Utils 2393SYNOPSIS 2394 xPK [-frsux] [-p password] [-m method] files 2395 2396 -m = packing method 2397 -f = force repack 2398 -s = do not remove original 2399 -r = recursively (un)pack 2400 -u = unpack (extract) 2401 -p = encrypt/decrypt 2402 -x = pack executables only 2403 2404DESCRIPTION 2405 xPK is a command line interface to the XPK compression library. 2406 It compresses a file using the method given by -m. After the 2407 process is complete, the original file is removed and replaced 2408 by its compressed version under the same name. 2409 2410 The xPK executable can be renamed to a packer name which will 2411 then be considered as given by -m. 2412 2413OPTIONS 2414 -m = method. After -m you can indicate the name of the packer 2415 to use, plus a mode number if the packer supports that. 2416 -f = force. Will enforce packing of already XPK-packed files. 2417 -s = suffix. Add a .XPK suffix to the compressed version and 2418 do not remove the original. 2419 -r = recur. If any directories are encountered, they are packed 2420 recursively. 2421 -u = unpack. Will unpack the indicated files. (same as -e). 2422 -p = password. Will be used for encryption or decryption. 2423 -x = executables. Will refuse to pack files that are not 2424 executable or are overlaid. For use with xLoadSeg. 2425 2426EXAMPLES 2427 xPK -rm NUKE dh1:modules 2428 xPK -m IMPL.50 df0:OVERVIEW 2429 xPK -xm NUKE dh4: 2430 xPK -r -p topsecret -m FEAL.32 dh1:private_docs 2431 2432COPYRIGHT 2433 Freely distributable for noncommercial use. 2434 2435AUTHOR 2436 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 2437@endnode 2438 2439@node xQuery 2440@toc C-Utils 2441SYNOPSIS 2442 xQuery 2443 xQuery [packer] 2444 2445DESCRIPTION 2446 xQuery shows important parameters about a packer, or if 2447 none indicated, all packers. 2448 2449EXAMPLE 2450 xQuery FEAL 2451 2452 Packer : FEAL 2453 Name : Fast Encryption ALgorithm 1.0 2454 Descr. : FEAL-N with CBC1. Password protects data with selectable safety. 2455 DefMode: 16 2456 Mode : 0..4 5..8 9..16 17..32 33..100 2457 Descr. : fastest fast safe safer safest 2458 PkSpeed: 238 K/s 171 K/s 109 K/s 63 K/s 34 K/s 2459 UpSpeed: 244 K/s 174 K/s 109 K/s 63 K/s 34 K/s 2460 Ratio : 0 % 0 % 0 % 0 % 0 % 2461 2462 The meaning of the fields: 2463 2464 Packer : The 4-letter name of the packer 2465 Name : The full packer name 2466 Descr. : The packer description 2467 DefMode: The default mode 2468 Mode : Below information is valid for this range of modes 2469 Descr. : Mode description 2470 PkSpeed: Packing speed for this mode range 2471 UpSpeed: Unpacking speed for this mode range 2472 Ratio : Compression factor (higher=better) 2473 2474 All timings were measured on an A3000. Divide by 5 to get 2475 timings for 68000. 2476 2477COPYRIGHT 2478 Freely distributable for noncommercial use. 2479 2480AUTHOR 2481 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 2482@endnode 2483 2484@node xType 2485@toc C-Utils 2486SYNOPSIS 2487 xType filenames 2488 2489DESCRIPTION 2490 Prints the given files to standard output, decompressing them if 2491 they are compressed. 2492 2493EXAMPLE 2494 xType intuition.doc.nuke 2495 2496COPYRIGHT 2497 Freely distributable for noncommercial use. 2498 2499AUTHOR 2500 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 2501@endnode 2502 2503@node xUp 2504@toc C-Utils 2505SYNOPSIS 2506 xUp [-s] [-S] [-p password] filenames 2507 2508DESCRIPTION 2509 xUp unpacks the given files, replacing the original by the 2510 uncompressed version. Wild cards are not supported, and file 2511 attributes are not yet preserved. 2512 When the decrunching fails, the source file is preserved. 2513 2514OPTIONS 2515 -s = suffix. Keeps compressed version (when suffix is added), 2516 stores uncompressed version under the same name minus .xpk 2517 suffix. 2518 -S = Same as -s, but removes any .#? suffix and not only .xpk one. 2519 -p = password. Uses given password for decompression 2520 2521EXAMPLE 2522 xUp -p secret mytext 2523 2524COPYRIGHT 2525 Freely distributable for noncommercial use. 2526 2527AUTHOR 2528 @{"Dirk St�cker" LINK "Contact Dirk St�cker"} 2529@endnode 2530 2531@node xScan 2532@toc C-Utils 2533SYNOPSIS 2534 xScan FILE/M/A,ALL/S,REMOVE/S 2535 2536DESCRIPTION 2537 xScan is a small CLI command which scans through XFH partition and 2538 modifies them. After those modifications XFH (V1.34 or newer) will 2539 be able to read directories MUCH faster. In fact you will no more 2540 notice a speed difference between XFH and the normal FileSystem. 2541 2542 VERY IMPORTANT: 2543 xScan will NOT work if you use it directly on a XFH partition, it 2544 will just do nothing. Instead of that you must use it on the 2545 PHYSICAL directory of the XFH partition. E.g. if your XFH partition 2546 is called "XH0:" and the rootdir of is "DH0:Archive", DO NOT use 2547 "xScan XH0: ALL" but "xScan DH0:Archive ALL". 2548 2549THEORY 2550 How does "xScan" work ? 2551 2552 If XFH scans through a directory it opens EVERY file to check if 2553 it is packed or not. That's why it is so slow. 2554 xScan scan once through the directories for files. If it finds one 2555 with an UNUSED filenote it adds a special one (filenote) to the file. 2556 This filenote contains an ID string, some check values and the length 2557 of the unpacked file(*). 2558 If the new XFH scans through the directories it checks for such 2559 filenotes and after finding one with still valid check values it will 2560 take the unpacked length from the filenote without opening the file. 2561 That's why the new version is faster. Of course these special 2562 filenotes will be hidden. 2563 2564 (*) I do not want to explain the format exactly, because people 2565 should not use these informations. 2566 2567ARGUMENTS 2568 FILE: You can supply as many files, directories or patterns 2569 as you want. 2570 2571 ALL: scan through directory trees 2572 2573 REMOVE: remove filenotes instead of creating them 2574 2575 Examples: 2576 2577 xScan SYS:Archive/MetaFont ALL 2578 2579 or 2580 2581 xScan Docs/#?.doc REMOVE 2582 2583HISTORY 2584 1.0 initial release for XFH 1.34 2585 1.1 adds special filenotes to unpacked files, too 2586 1.2 does NOT follow softlinks any more 2587 2588COPYRIGHT 2589 xScan is free to be spread on public-domain and shareware disks as 2590 long as they are sold for a reasonable charge that is less than $6. 2591 This applies not to Fred Fish, he and ONLY he can take more money. 2592 For use in commercial products the permission of the author is 2593 required. 2594 2595AUTHOR 2596 @{"Matthias Scheler" LINK "Contact Matthias Scheler"} 2597@endnode 2598 2599@node Contacts 2600Please remember, that some of these addresses may be false, so do not 2601blame, if you do not get answer. If you get newer information, please 2602contact me (the first one). 2603 2604Autors of the main xpkmaster system (and some additional stuff). 2605Contact in the given order! 2606 2607 @{"Dirk St�cker" LINK "Contact Dirk St�cker"} 2608 @{"Christian von Roques" LINK "Contact Christian von Roques"} 2609 @{"Urban Dominik M�ller" LINK "Contact Urban Dominik M�ller"} 2610 @{"Bryan Ford" LINK "Contact Bryan Ford"} 2611 2612Autors of Sublibraries: 2613 2614 Andr� Beck @{"IDEA" LINK IDEA} 2615 @{"Karsten Dagef�rde" LINK "Contact Karsten Dagef�rde"} @{"RAKE" LINK RAKE} 2616 @{"Stephan Fuhrmann" LINK "Contact Stephan Fuhrmann"} @{"DLTA" LINK DLTA} 2617 @{"Martin Hauner" LINK "Contact Martin Hauner"} @{"HFMN" LINK HFMN} 2618 @{"John Hendrikx" LINK "Contact John Hendrikx"} @{"SQSH" LINK SQSH} 2619 @{"Zdenek Kabelac" LINK "Contact Zdenek Kabelac"} @{"MASH" LINK MASH} 2620 @{"Jorma Oksanen" LINK "Contact Jorma Oksanen"} @{"SMPL" LINK SMPL}, FRLE 2621 @{"Christian von Roques" LINK "Contact Christian von Roques"} @{"FAST" LINK FAST}, @{"FEAL" LINK FEAL} 2622 @{"Peter Struijk" LINK "Contact Peter Struijk"} @{"IMPL" LINK IMPL} 2623 @{"Marc Zimmermann" LINK "Contact Marc Zimmermann"} @{"HUFF" LINK HUFF} 2624 2625Translators: 2626 catal� Lloren� Grau <llg@cryogen.com> 2627 Dansk Jacob Laursen <laursen@myself.com> 2628 Thomas L. Petersen <thomaslp@post1.tele.dk> 2629 Deutsch @{"Dirk St�cker" LINK "Contact Dirk St�cker"} 2630 Espa�ol D�maso D. Est�vez <amidde@arrakis.es> 2631 Fran�ais Georges 'Melkor' Goncalves <melkor@lords.com> 2632 Laurent Kemp� <lkempe@nucleus.fr> 2633 Hrvatski Mladen Ilisinovic <milisino@public.srce.hr> 2634 Italiano Dario Manzoni <dmanzoni@spin.it> 2635 Nederlands Frits Letteboer <frits.letteboer@hetnet.nl> 2636 Leon Woestenberg <leon@stack.nl> 2637 Norsk Kim Roar Utsi <kimme@arcticnet.no> 2638 Polski Marcin Orlowski <carlos@amiga.com.pl> 2639 Portugu�s R�ben Alvim <mindwalker@mail.telepac.pt> 2640 Frederico Borges <famb@mail.telepac.pt> 2641 Russian Oleg Sergeev <bigblack@neworder.spb.ru> 2642 Srpski Ljubomir Jankovi <lurch@beotel.yu> 2643 Andrija Antonijevic <TheAntony@bigfoot.com> 2644 Suomi Pekka Kolehmainen <pekkak@icenet.fi> 2645 Mika Lundell <c71829@uwasa.fi> 2646 Svenska Jon �slund <jooon@hem1.passagen.se> 2647 Mattias Gustafsson 2648 Andreas P�lsson <did@algonet.se> [version 3.11] 2649 �e�tina Vit Sindlar <sindlar@jackal.cis.vutbr.cz> 2650 2651 A lot thanks also to Marcin Orlowski of Amiga Translators' Organization 2652 <http://ato.home.pages.de/>, who manages translation stuff. 2653 2654Other related persons: 2655 2656 @{"Matthias Meixner" LINK "Contact Matthias Meixner"} xpkarchive.library, @{"SHRI" LINK SHRI} 2657 @{"Kristian Nielsen" LINK "Contact Kristian Nielsen"} XFH 2658 @{"Nicola Salmoria" LINK "Contact Nicola Salmoria"} XFH commodity 2659 @{"Matthias Scheler" LINK "Contact Matthias Scheler"} XFH, @{"xPack" LINK xPack} 2660 @{"Christian Schneider" LINK "Contact Christian Schneider"} XPK concept, @{"xLoadSeg" LINK xLoadSeg} 2661 @{"Jan Schwenke" LINK "Contact Jan Schwenke"} HotHelp files 2662 2663And surely there are a lot of persons I forgot. 2664@endnode 2665 2666@node "Contact Dirk St�cker" 2667@toc Contacts 2668Name: Dirk St�cker 2669Address: Geschwister-Scholl-Stra�e 10 2670 01877 Bischofswerda 2671 GERMANY 2672Telephone: GERMANY (+49) (0)3594 706666 2673E-Mail: stoecker@amigaworld.com 2674 dstoecker@gmx.de 2675WWW: http://home.pages.de/~Gremlin/ 2676 http://www.amigaworld.com/support/xpkmaster/ 2677@endnode 2678 2679@node "Contact Christian von Roques" 2680@toc Contacts 2681Name: Christian von Roques 2682Address: Forststrasse 71 2683 76131 Karlsruhe 2684 GERMANY 2685Telephone: GERMANY (+49) (0)721 621253 2686or 2687Address: Kastanienweg 4 2688 78713 Schramberg 2689 GERMANY 2690Telephone: GERMANY (+49) (0)7422 53822 2691E-Mail: roques@pond.sub.org 2692 roques@ipd.info.uni-karlsruhe.de 2693 roques@ira.uka.de 2694@endnode 2695 2696@node "Contact Bryan Ford" 2697@toc Contacts 2698Name: Bryan Ford 2699Address: 8749 Alta Hills Circle 2700 Sandy, UT 84093 2701 USA 2702Telephone: (801) 585-4619 2703E-Mail: bryan.ford@m.cc.utah.edu 2704 baf0863@cc.utah.edu 2705 baf0863@utahcca.bitnet 2706@endnode 2707 2708@node "Contact Urban Dominik M�ller" 2709@toc Contacts 2710Name: Urban Dominik M�ller 2711Address: Schulhausstrasse 83 2712 CH-6312 Steinhausen 2713 SWITZERLAND 2714E-Mail: umueller@indiac.relog.ch 2715 umueller@amiga.icu.net.ch 2716 umueller@amiga.physik.unizh.ch 2717 umueller@iiic.ethz.ch 2718@endnode 2719 2720@node "Contact Karsten Dagef�rde" 2721@toc Contacts 2722Name Karsten Dagef�rde 2723E-Mail: dagefoer@rzcipa03.rz.tu-bs.de 2724 dagefoer@ibr.cs.tu-bs.de 2725 dagefoer@rob.cs.tu-bs.de 2726 K.Dagefoerde@tu-bs.de 2727@endnode 2728 2729@node "Contact Stephan Fuhrmann" 2730@toc Contacts 2731Name: Stephan Fuhrmann 2732Address: Ostmarkstra�e 19 2733 76227 Karlsruhe 2734 GERMANY 2735E-Mail: Stephan.Fuhrmann@stud.uni-karlsruhe.de 2736@endnode 2737 2738@node "Contact Martin Hauner" 2739@toc Contacts 2740Name: Martin Hauner 2741Address: Max-Born-Stra�e 5 2742 38116 Braunschweig 2743 GERMANY 2744E-Mail: drizzt@trashcan.escape.de 2745@endnode 2746 2747@node "Contact John Hendrikx" 2748@toc Contacts 2749Name: John Hendrikx 2750Address: Figarostraat 36 2751 3208 PD Spijkenisse 2752 NETHERLANDS 2753E-Mail: FIDO: 2:285/813.8 2754 AMY: 39:153/201.8 2755 NLA: 14:101/200.8 2756@endnode 2757 2758@node "Contact Zdenek Kabelac" 2759@toc Contacts 2760Name: Zdenek Kabelac 2761Address: Policna 135 2762 75701 Valasske Mezirici 2763 Czech Republic 2764E-Mail: kabi@informatics.muni.cz 2765WWW: http://www.muni.cz/~kabi/ 2766@endnode 2767 2768@node "Contact Jorma Oksanen" 2769@toc Contacts 2770Name: Jorma Oksanen 2771Address: H�meentie 6-8 A 4 2772 13200 H�MEENLINNA 2773 FINLAND 2774E-Mail: tenu@sci.fi 2775Telephone: FINLAND (+358) (3) 6120 217 2776@endnode 2777 2778@node "Contact Jan Schwenke" 2779@toc Contacts 2780Name: Jan Schwenke 2781Address: Dorfstra�e 55 2782 09465 Cranzahl 2783 GERMANY 2784E-Mail: jsc@fh-zwickau.de 2785@endnode 2786 2787@node "Contact Peter Struijk" 2788@toc Contacts 2789Name: Peter Struijk 2790Address: Veulenkamp 28 2791 2623 XD DELFT 2792 NETHERLANDS 2793E-Mail: winfjmf@dutiws.twi.tudelft.nl 2794@endnode 2795 2796@node "Contact Marc Zimmermann" 2797@toc Contacts 2798Name: Marc Zimmermann 2799E-Mail: zimmerma@ibr.cs.tu-bs.de 2800@endnode 2801 2802@node "Contact Matthias Meixner" 2803@toc Contacts 2804Name: Matthias Meixner 2805Address: Sandberg 13 2806 36145 Schwarzbach 2807 GERMANY 2808E-Mail: meixner@rbg.informatik.th-darmstadt.de 2809@endnode 2810 2811@node "Contact Kristian Nielsen" 2812@toc Contacts 2813Name: Kristian Nielsen 2814Address: Groenjordskollegiet 2815 room 6111 2816 Groenjordsvej 2817 DK-2300 Koebenhavn S 2818 DENMARK 2819E-Mail: bombadil@diku.dk 2820@endnode 2821 2822@node "Contact Nicola Salmoria" 2823@toc Contacts 2824Name: Nicola Salmoria 2825Address: Via Piemonte 11 2826 53100 Siena 2827 ITALY 2828E-Mail: MC6489@mclink.it 2829@endnode 2830 2831@node "Contact Matthias Scheler" 2832@toc Contacts 2833Name: Matthias Scheler 2834Address: Sch�tzenstra�e 18 2835 33178 Borchen 2836 GERMANY 2837Telephone: GERMANY (+49) (0)5251 399031 2838E-Mail: tron@lyssa.pb.owl.de 2839 FidoNet: Matthias Scheler 2:243/6310.10 2840@endnode 2841 2842@node "Contact Christian Schneider" 2843Name: Christian Schneider 2844Address: Im Schilf 15 2845 CH-8044 Zurich 2846 SWITZERLAND 2847E-Mail: BIX: hschneider 2848 Internet: cschneid@amiga.physik.unizh.ch 2849@endnode 2850 2851