1@external-css=allegro.css 2@document_title=Datafile format 3 4<center><h1><b> 5Datafiles format information. 6</b></h1></center> 7<hr> 8 9 10@!text 11@heading 12Contents 13 14@shortcontents 15 16 17@text 18 19@heading 20Introduction 21 22An Allegro datafile is a bit like a zip file in that it consists of lots of 23different pieces of data stuck together one after another, and optionally 24compressed. This means that your game doesn't have to clutter up the disk 25with hundreds of tiny files, and it makes programming easier because you can 26load everything with a single function call at program startup. Another 27benefit is that the LZSS file compression algorithm works much better with 28one large file than with many small ones. 29 30Datafiles have the extension .dat, and can be created and edited with the 31graphical grabber program or the command line dat utility. They can be 32stored as separate files and loaded into memory by the load_datafile() 33function, or you can use dat2s to convert them into asm code which can then 34be linked directly into your executable. 35 36@heading 37Objects 38 39Each datafile contains a number of objects, of varying types. Object types 40are represented by 32 bit integer ID's, which are interpreted as four 41character ASCII strings. These ID's can be constructed with the DAT_ID() 42macro, for example a DATA object is represented by DAT_ID('D','A','T','A'), 43or you can use the predefined DAT_* constants for the standard data types: 44 45<ul><li> 46DAT_FILE - "FILE"<br> 47 A datafile, which contains a list of other objects. Datafile objects 48 can be nested inside other datafiles, allowing you to create 49 hierarchical structures of any depth. 50 51<li>DAT_DATA - "DATA"<br> 52 A block of binary data. Allegro treats all unknown types as binary 53 data objects, so you don't need to use this ID: you can create custom 54 object formats using whatever ID's you like. 55 56<li>DAT_FONT - "FONT"<br> 57 A font. 58 59<li>DAT_SAMPLE - "SAMP"<br> 60 A digital sound sample. 61 62<li>DAT_MIDI - "MIDI"<br> 63 A MIDI file. 64 65<li>DAT_PATCH - "PAT "<br> 66 A Gravis patch (MIDI instrument). 67 68<li>DAT_FLI - "FLIC"<br> 69 An FLI or FLC animation. 70 71<li>DAT_BITMAP - "BMP "<br> 72 A bitmap. 73 74<li>DAT_RLE_SPRITE - "RLE "<br> 75 A run length encoded sprite. 76 77<li>DAT_C_SPRITE - "CMP "<br> 78 A compiled sprite. 79 80<li>DAT_XC_SPRITE - "XCMP"<br> 81 A mode-X compiled sprite. 82 83<li>DAT_PALETTE - "PAL "<br> 84 A 256 color palette. 85 86<li>DAT_PROPERTY - "prop"<br> 87 An object property (see below). You will never directly encounter this 88 object type, but you should be aware that it is treated specially by 89 the datafile code. 90 91<li>DAT_INFO - "info"<br> 92 The grabber utility uses this object to store information about the 93 datafile. Like property objects, you ought never to encounter it, but 94 you should avoid using the ID for any custom object formats you create. 95 96<li>DAT_END - -1<br> 97 Special marker used to indicate the end of a datafile. 98</ul> 99 100@heading 101Properties 102 103Each object can have any number of properties attached to it. These are 104ASCII strings describing attributes of the object, such as its name and 105where it came from. Like the objects themselves, properties are identified 106by 32 bit integer ID's which are constructed from four character strings by 107the DAT_ID() macro. Allegro defines the standard properties: 108 109<ul><li> 110"NAME"<br> 111 The name of the object. 112<li> 113"ORIG"<br> 114 The object's origin, ie. the name of the file from which it was 115 grabbed. 116<li> 117"DATE"<br> 118 A timestamp, used by the update command in the grabber and dat 119 utilities. This is the modification time of the file from which the 120 object was grabbed, in "m-dd-yyyy, hh:mm" format. 121<li> 122"XPOS"<br> 123 For bitmap objects which were grabbed from part of a larger image, the 124 x position of the origin within the parent bitmap. 125<li> 126"YPOS"<br> 127 For bitmap objects which were grabbed from part of a larger image, the 128 y position of the origin within the parent bitmap. 129<li> 130"XSIZ"<br> 131 For bitmap objects which were grabbed from part of a larger image, the 132 width of the selected region. 133<li> 134"YSIZ"<br> 135 For bitmap objects which were grabbed from part of a larger image, the 136 height of the selected region. 137<li> 138"XCRP"<br> 139 For autocropped bitmap objects, the amount of cropping on the left of 140 the image. 141<li> 142"YCRP"<br> 143 For autocropped bitmap objects, the amount of cropping at the top of 144 the image. 145</ul> 146 147You can use whatever other ID's you like to store custom information about 148your objects (the grabber internally use some other properties stored in a 149hidden DAT_INFO object, so they won't conflict with yours). 150 151 152 153@heading 154File format specification 155 156In case anyone wants to do some serious hackery, and for my own future 157reference, here are some details of the innards of the datafile format. 158 159Note that this is different to the datafile format used by Allegro versions 1602.1 and earlier. Allegro can still load files from the old format, but it 161was much less flexible and didn't support nice things like object 162properties, so you should load any old files into the grabber and save them 163out again to convert to the new format. 164 165Nb. if all you want to do is write a utility that manipulates datafiles in 166some way, the easiest approach is probably to use the helper functions in 167datedit.c, which are currently shared by the dat, dat2s, and grabber 168programs. These functions handle loading, saving, inserting and deleting 169objects, and modifying the contents of datafiles in various ways, but life 170is too short for me to bother documenting them all here. Look at the 171source... 172 173Anyway. All numbers are stored in big-endian (Motorola) format. All text is 174stored in UTF-8 encoding. A datafile begins with one of the 32 bit values 175F_PACK_MAGIC or F_NOPACK_MAGIC, which are defined in allegro.h. If it starts 176with F_PACK_MAGIC the rest of the file is compressed with the LZSS 177algorithm, otherwise it is uncompressed. This magic number and optional 178decompression can be handled automatically by using the packfile functions 179and opening the file in F_READ_PACKED mode. After this comes the 32 bit 180value DAT_MAGIC, followed by the number of objects in the root datafile (not 181including objects nested inside child datafiles), followed by each of those 182objects in turn. 183 184Each object is in the format: 185 186<textblock> 187 OBJECT = 188 var - <property list> - any properties relating to the object 189 32 bit - <type ID> - object type ID 190 32 bit - <compressed size> - size of the raw data in the file 191 32 bit - <uncompressed size> - see below 192 var - <data> - the contents of the object 193<endblock> 194 195The property list can contain zero or more object properties, in the form: 196 197<textblock> 198 PROPERTY = 199 32 bit - <magic> - "prop" 200 32 bit - <type ID> - property type ID 201 32 bit - <size> - size of the property string, in bytes 202 var - <data> - property string, _not_ null-terminated 203<endblock> 204 205If the uncompressed size field in an object is positive, the contents of the 206object are not compressed (ie. the raw and compressed sizes should be the 207same). If the uncompressed size is negative, the object is LZSS compressed, 208and will expand into -<uncompressed size> bytes of data. The easiest way to 209handle this is to use the pack_fopen_chunk() function to read both the raw 210and compressed sizes and the contents of the object. 211 212The contents of an object vary depending on the type. Allegro defines the 213standard types: 214 215<textblock> 216 DAT_FILE = 217 32 bit - <object count> - number of objects in the sub-file 218 var - <object list> - objects in the same format as above 219 220 DAT_FONT = 221 16 bit - <font size> - 8, 16, -1, or 0 222 223 if font size == 8 { - obsolete as of version 3.9.x! 224 unsigned char[95][8] - 8x8 bit-packed font data 225 } 226 227 if font size == 16 { - obsolete as of version 3.9.x! 228 unsigned char[95][16] - 8x16 bit-packed font data 229 } 230 231 if font size == -1 { - obsolete as of version 3.9.x! 232 95x { 233 16 bit - <width> - character width 234 16 bit - <height> - character height 235 var - <data> - character data (8 bit pixels) 236 } 237 } 238 239 if font size == 0 { - new format introduced in version 3.9.x 240 16 bit - <ranges> - number of character ranges 241 for each range { 242 8 bit - <mono> - 1 or 8 bit format flag 243 32 bit - <start> - first character in range 244 32 bit - <end> - last character in range (inclusive) 245 for each character { 246 16 bit - <width> - character width 247 16 bit - <height> - character height 248 var - <data> - character data 249 } 250 } 251 } 252 253 DAT_SAMP = 254 16 bit - <bits> - sample bits (negative for stereo) 255 16 bit - <freq> - sample frequency 256 32 bit - <length> - sample length 257 var - <data> - sample data 258 259 DAT_MIDI = 260 16 bit - <divisions> - MIDI beat divisions 261 32x { 262 32 bit - <length> - track length, in bytes 263 var - <data> - MIDI track data 264 } 265 266 DAT_FLI = 267 var - <data> - FLI or FLC animation, standard format 268 269 DAT_BITMAP = 270 DAT_C_SPRITE = 271 DAT_XC_SPRITE = 272 16 bit - <bits> - bitmap color depth 273 16 bit - <width> - bitmap width 274 16 bit - <height> - bitmap height 275 var - <data> - bitmap data 276 277 Valid color depths are 8, 15, 16, 24, 32, and -32. Both 15 and 16 bit 278 images are stored in 5.6.5 RGB format, and 24 and 32 bit images as 279 8.8.8 RGB. The special -32 flag indicates that the data is in true 32 280 bit RGBA format. 281 282 DAT_RLE_SPRITE = 283 16 bit - <bits> - sprite color depth 284 16 bit - <width> - sprite width 285 16 bit - <height> - sprite height 286 32 bit - <size> - data size, in bytes 287 var - <data> - RLE compressed sprite data 288 289 Valid color depths are 8, 15, 16, 24, 32. and -32. Both 15 and 16 bit 290 images are stored in 5.6.5 RGB format with 16 bit skip counts and EOL 291 markers, and 24 and 32 bit images as 8.8.8 RGB. with 32 bit skip 292 counts and markers. The special -32 flag indicates that the data is in 293 true 32 bit RGBA format. 294 295 DAT_PALETTE = 296 256 x { 297 8 bit - <red> - red component, 0-63 298 8 bit - <green> - green component, 0-63 299 8 bit - <blue> - blue component, 0-63 300 8 bit - <pad> - alignment padding 301 } 302<endblock> 303 304I think that covers everything. 305