1 FILE SERVICE PROTOCOL VERSION 2 2 OFFICIAL PROTOCOL DEFINITION 3 FSP v2 4 5 Document version 0.19 6 Last updated 01 Oct 2009 7 8 Also known as 9 File Slurping Protocol, 10 Flaky Stream Protocol, 11 FTP's Sexier Partner, 12 File Sharing Protocol or 13 Fucking Slow Protocol. 14 15 `FSP is what anonymous FTP *should* be' 16 17This document was created by Radim Kolar, because there is no RFC for 18FSP. It was planed, but never comes out. See FSP Project Home page 19http://fsp.sourceforge.net/ for up-to-date version of this document. 20Also contact Radim Kolar with questions and if you need help with an 21implementation of this protocol in productivity environment. 22 23This document is not copyrighted and is placed into public domain. 24 25Data formats used in this document: 26 27 byte - unsigned 1 byte integer 0-255 28 word - unsigned 2 byte integer 0-65535 29 long - unsigned 4 byte integer 0-4294967295 30 bits - counted from right to left. bit 0 is the lowest. 31 NULL - byte 00 32 ASCIIZ - ASCII string terminated with one NULL (the same as used in C 33 language) 34 Numbers are stored packed in network byte order (high byte first). 35 hexadecimal (base 16) numbers have 0x prefix. 36 File or directory names uses '/' as directory separator, they do not 37 need to start or end with it. There are in ASCIIZ format. 38 FSP servers starting from version 2.8 can have optional password 39 protection. To get password protected file, append '\n' followed 40 by password to filename. 41 42Uniform Resource Locator 43 44 Uniform resource locator format for FSP v2 protocol is: 45 46 fsp://password@hostname:port/directory/filename.ext 47 48 If port is omitted FSP standard port 21 is used, if password 49 part is omitted no password is sent. If no password is used URL must 50 not contain @ character before hostname. FSP URL do not have query 51 and anchor parts. 52 53Transport 54 55 FSP uses UDP datagrams as standard transport medium for operation in 56 Internet Networks. 57 58 FSP datagram header has checksum and payload size recorded. Because 59 of this FSP do not require any underlying transport protocol on 60 layer 2 and can be used as very simple raw-protocol (for example for 61 sending data over serial line). This makes it very popular in 62 embedded devices area, because it is extremely easy to implement. 63 64 FSP packets can have an optional extra data area. For supporting 65 packets with this, underlying transport must make size of received 66 packet available to FSP protocol stack at server side. Without this 67 information, full support for extra data area is not possible. 68 Partial support for extra data is still possible: Some 69 commands are carrying size of extra data in file position field and 70 client side checksums contains total packet size. Using checksums 71 for decoding of length of extra data is last resort, because 72 checksums are only one byte in size. 73 74 Servers can still send extra data in reply to CC_VERSION and 75 CC_GET_PRO because length of extra data is recorded in position 76 field of FSP header. Clients must check this field, when checking 77 checksums of received packets of that kind and process these extra 78 bytes. 79 80 Minimum FSP packet size (not including size of UDP, IP and link 81 layer headers) is 12 bytes (FSP v2 header only), maximum standard 82 FSP packet size is 12 bytes of FSP v2 header and 1024 bytes of 83 payload. Server can optionally accept longer packets, but must not 84 send longer packets to client unless specially requested. All 85 servers and clients must support receiving 1024+12 bytes long 86 packets. 87 88Security 89 90 Design 91 92 FSP protocol was not designed to transfer secret data. It was 93 designed as alternative protocol for providing lightweight access to 94 collection of public files. FSP has the same user level of security 95 as the anonymous FTP file server. FSP has better network level 96 security, because it was designed to resist various 97 denial-of-service attacks. FSP protocol v3 will be designed to be 98 fully secure. FSP3 will not be backward compatible with FSP2. 99 100 Passwords 101 102 Recently, password support was added to protocol, because just too 103 many people wants it and there were hacked version of FSP with 104 password support floating around. Passwords are transmitted in clear 105 text over network which makes them a weak protection for determined 106 intruder when transferred over unsecured network. 107 108FSP Packet format: 109 HEADER - size = Fixed size 12 bytes. Always present. 110 DATA - size = defined in header (DATA_LENGTH) 111 XTRA DATA - size = packet_size - header_size (12) - DATA_LENGTH 112 113Size of data payload is DATA_LENGTH + XTRA_DATA length. Clients and 114servers are not required to support XTRA DATA (but current FSP 115implementation does). If XTRA DATA are provided, there must be also 116contained in MESSAGE_CHECKSUM. 117 118FSP v2 HEADER FORMAT (12 bytes) 119 byte FSP_COMMAND 120 byte MESSAGE_CHECKSUM 121 word KEY 122 word SEQUENCE 123 word DATA_LENGTH 124 long FILE_POSITION 125 126MESSAGE_CHECKSUM 127Entire packet (HEADER + DATA + XTRA DATA) is checksummed. When computing a 128checksum use zero in place of MESSAGE_CHECKSUM header field. 129 130Method of computing checksums is different in each direction. For 131packets traveling from server to client initial checksum value is 132zero, otherwise initial value is packet size (HEADER + DATA + XTRA DATA). 133To get more information why checksums are different in each 134direction see comment about optional extra data in Transport section. 135 136Checksums in server->client direction are computed as follows: 137 138 /* assume that we have already zeroed checksum in packet */ 139 unsigned int sum,checksum; 140 unsigned char *t; 141 for(t = packet_start, sum = 0; t < packet_end; sum += *t++); 142 checksum= sum + (sum >> 8); 143 144One byte checksums can be considered weak when compared with other 145protocols, which are using at least CRC16 checksum types. FSP server 146fed by random data can resist for hours without falsely accepting 147random data as valid FSP packet. This demonstration shows, that these 148checksums when very easy to compute, are sufficient for guarding 149against random line noise. 150 151Note: 152IP/UDP packet has its own CRC16 checksum, but FSP protocol checksum is 153used as protection against received non FSPv2 UDP packets. 154 155KEY 156Client's message to server contain a KEY value that is the same as the KEY 157value of the previous message received from the server. KEY is chosen random 158by server. 159 160Server uses one KEY per client network address. If multiple FSP clients 161on the same host wants access same FSP server at the same time, they 162must implement some local method of key exchange. If they don't server 163will serve only one client from host, because other clients do not 164have valid key. 165 166TIMEOUTS 1671. Resend 168 169Server will accept resent message from client with old KEY after 3 170seconds. Client MUST wait at least 1 second before resending a 171message. It is recommended to use initial delay of 1.34 second and 172after each unsuccessful resend multiply delay time by 1.5. Maximum 173delay time is 300 seconds. Recommended maximum delay between resend 174is 60 seconds. 175 1762. Session 177 178Server will accept message with bad key after 60 seconds. Clients 179should sent CC_BYE at end of their session because CC_BYE terminates a 180session. After session is terminated, sever will accept any next key. 181 182SEQUENCE 183Similarly, the server's message to client contains a SEQUENCE value 184that is the same as the SEQUENCE value of the previous message from the client. 185Client can choose any SEQUENCE number and can use it for detection of lost 186packets (increase sequence number on message resend). 187 188DATA_LENGTH 189Size of DATA field in packet. Packet can also contain XTRA DATA field but 190size of this field is not included in header and must be computed from 191received packet size or from knowledge of FSP v2 packets formats (some 192packets carries length of extra data in position field). 193 194FILE POSITION 195When transferring files, this field shows current position of requested 196data. 197 198FSP COMMANDS 199============ 200 201REQUIRED COMMANDS 202FSP File servers MUST supports following commands: 203- sending error messages back to client with CC_ERR 204- directory listings CC_GET_DIR 205- file transfer CC_GET_FILE 206- file status CC_STAT 207- information about directory flags CC_GET_PRO 208- terminate session CC_BYE 209If server supports packets with payload size over 1024 bytes, supporting 210CC_VERSION is recommended. 211 212 213 CC_VERSION 0x10 - Get server version string and setup 214 215 request 216 file position: ignored 217 data: not used 218 xtra data: not used 219 220 reply 221 file position: size of optional extra version data 222 data: ASCIIZ Server version string 223 xtra data: optional extra version data 224 byte - FLAGS 225 bit 0 set - server does logging 226 bit 1 set - server is read only 227 bit 2 set - reverse lookup required 228 bit 3 set - server is in private mode 229 bit 4 set - thruput control 230 bit 5 set - server accept XTRA 231 DATA on input 232 233 if bit 4 is set thruput info follows 234 long - max_thruput allowed (in bytes/sec) 235 word - max. payload size supported by server 236 if > 1024, otherwise preferred 237 payload size. 238 239 Compatibility 240 241 Max. / preferred packet size supported is reported only 242 by fspd 2.8.1 b20 or newer. 243 244 Bit 5 - accept xtra data flag is set only by fspd 245 2.8.1 b21 or newer. 246 247 Note 248 249 Some FSP servers do not responds to this command, 250 because this command is used by FSP scanners and 251 servers do not wishes to be detected. 252 253 CC_ERR 0x40 - error response from server 254 255 If you want to get a error from server, send 256 any unknown client command (for example CC_ERR). 257 CC_ERRs are normally sent only by server on 258 errors conditions. 259 260 request (not used) 261 file position: not used 262 data: not used 263 xtra data: not used 264 265 reply 266 file position: size of extra data 267 data: ASCIIZ Error string 268 xtra data: not required 269 word - error status code 270 271 Error status codes are not currently standardised. If you 272 wish to participate in standardization process join FSP-devel 273 mailing list. For use in your own software use vendor-specific 274 error codes which have reserved range 0xF000 - 0xFFFF. 275 276 Compatibility 277 278 Support for sizing of extra data in reply was added 279 in fsp 2.8.1b26. Previous versions left this field 280 unchanged. In old versions it will most likely contain 281 file position used by requests but not always. In some 282 code paths in fspd it will be just uninitialized memory. 283 Its recommended to check if file position in reply is 2 284 otherwise it should be processed as containing zero. 285 286 CC_GET_DIR 0x41 - get a directory listing 287 288 request 289 file position: position in directory 290 data: ASCIIZ directory name 291 xtra data: (not required) 292 word - preferred size of directory block 293 294 reply 295 file position: same as in request 296 data: directory listing (format follows) 297 xtra data: not used 298 299Directory listing is transferred in similar way as file transfer. 300Directory listing is divided into blocks of equal size, only exception 301is last block which can be shorter. Default size of directory listing 302block is 1024 bytes. Server can use preferred block size sent by client 303and split directory listing into blocks with size preferred by client. 304 305Directory blocks can't be split across message boundary and client 306can't do seeking to any arbitrary offset, which can broke dirblock 307into 2 messages. In short: Every message must contain only one 308unsplited directory block. 309 310RDIRENT is the structure of a directory entry contained in a directory listing. 311Each entry contains a HEADER, which has 4 bytes quantity 'time' in Unix 312standard format, a 4 bytes quantity 'size', and 1 byte of 'type'. Header is 313followed by ASCIIZ encoded 'name'. RDIRENT is followed by enough number of 314padding to fill to an 4-byte boundary. 315 316At this point, if the next RDIRENT entry to follow will spread across 317directory block boundary, then two possible things will happen: 318 319 1) if the HEADER fits between this entry and the directory block 320 boundary, a complete header will be filled in with a 'type' set to 321 RDTYPE_SKIP and no name followed - just pad to boundary. Clients 322 which sees RDTYPE_SKIP header skips over remaining data in packet. 323 324 2) if the HEADER does not fit, then simply pad to the directory 325 block boundary. 326 327 This will make sure that messages carrying directory information carry only 328 complete directory entries and no fragmented entries. 329 330The last entry has type RDTYPE_END. 331 332 struct RDIRENT { 333 struct HEADER { 334 long time; 335 long size; 336 byte type; 337 } 338 ASCIIZ name; 339 } 340 341RDIRENT.HEADER types: 342 RDTYPE_END 0x00 343 RDTYPE_FILE 0x01 344 RDTYPE_DIR 0x02 345 RDTYPE_SKIP 0x2A 346 347 If directory listing contains symlink server can encode symlink 348 as source\ndestination\0. File type is set to the destination of 349 symlink RDTYPE_FILE or RDTYPE_DIR. Servers are not required to 350 support this feature. 351 352 CC_GET_FILE 0x42 - get a file 353 354 request 355 file position: offset in file 356 data: ASCIIZ filename 357 xtra data: (not required) 358 word - preferred size of reply's data block 359 360 reply 361 file position: same as in request 362 data: binary file data 363 xtra data: not used 364 365 CC_UP_LOAD 0x43 - open a file for writing 366 367 request 368 file position: offset in file 369 data: binary file data 370 xtra data: not used 371 372 reply 373 file position: same as in request 374 data: not used 375 xtra data: not used 376 377 CC_INSTALL 0x44 - close and install file opened for writing 378 379 request 380 file position: length of extra data 381 data: ASCIIZ filename 382 xtra data: (not required) 383 long - timestamp in Unix format 384 385 reply 386 file position: not used 387 data: not used 388 xtra data: not used 389 390 To cancel upload in progress without installing any 391 file send CC_INSTALL command with zero length (only 00 392 terminator) filename. This removes temporary data 393 created by upload. 394 395 Compatibility 396 397 Upload cancel feature and sizing of extra data was 398 first used in fsp 2.8.1b22. 399 400 CC_DEL_FILE 0x45 - delete a file 401 402 request 403 file position: not used 404 data: ASCIIZ filename 405 xtra data: not used 406 407 reply 408 file position: not used 409 data: not used 410 xtra data: not used 411 412 CC_DEL_DIR 0x46 - delete a directory 413 414 request 415 file position: not used 416 data: ASCIIZ directory 417 xtra data: not used 418 419 reply 420 file position: not used 421 data: not used 422 xtra data: not used 423 424 CC_GET_PRO 0x47 - get directory protection 425 426 request 427 file position: not used 428 data: ASCIIZ directory 429 xtra data: (not required) 430 word - preferred size of reply's optional data 431 (used for readme) + xtra data 432 433 reply 434 file position: number of extra protection bytes (now 1) 435 data: ASCIIZ directory readme 436 xtra data: protection data (format follows) 437 438 Protection bits: 439 0 - caller owns the directory 0x01 440 1 - files can be deleted from this dir 0x02 441 2 - files can be added to this dir 0x04 442 3 - new subdirectories can be created 0x08 443 4 - files are NOT readable by non-owners 0x10 444 5 - directory contain an readme file 0x20 445 6 - directory can be listed 0x40 446 7 - files can be renamed in this directory 0x80 447 448 Compatibility 449 450 Versions older than 2.8.1b6 do not uses bits 6 and 7. This 451 causes that directory can be listable even if do not have 452 6th bit set. 453 454 CC_SET_PRO 0x48 - set directory protection 455 456 request 457 file position: size of extra data 458 data: ASCIIZ directory 459 xtra data: 2 bytes of protection change command 460 1st byte: 461 <'+'|'-'> set or remove protection 462 2nd byte: 463 <'c'|'d'|'g'|'m'|'l'|'r'> 464 c public can create files 465 d public can delete files 466 g public can get files 467 m public can create directories here 468 l public can list directory 469 r public can rename files 470 471 reply 472 same as CC_GET_PRO 473 474 Compatibility 475 FSP versions older than 2.8.1 beta15 used p flag instead 476 g flag. +p = -g 477 478 Sizing of extra data in request was added in 479 fsp2.8.1b22. 480 481 CC_MAKE_DIR 0x49 - create a directory 482 483 request 484 file position: not used 485 data: ASCIIZ directory name 486 xtra data: not used 487 488 reply 489 same as CC_GET_PRO 490 491 CC_BYE 0x4A - finish a session 492 request 493 file position: not used 494 data: not used 495 xtra data: not used 496 497 reply 498 file position: not used 499 optional data: not used 500 xtra data: not used 501 502 You should send this packet when you are done with 503 talking to server. This causes that server will 504 accept next packet from your IP with any key. 505 506Commands starting from FSP version 2.4 ( released March 27, 1992 ) 507 508 CC_GRAB_FILE 0x4B - atomic get+delete a file 509 510 same format as CC_GET_FILE, but file is deleted after 511 successful transfer is done. If there are multiple 512 grabs for the same file, only one will succeed. 513 514 CC_GRAB_DONE 0x4C - atomic get+delete a file done 515 516 same format as CC_INSTALL. File is not 517 installed, but deleted. 518 519Commands starting from FSP 2.8.1 Beta 11 520 521 CC_STAT 0x4D - get information about file/directory 522 523 request 524 file position: not used 525 data: ASCIIZ directory or file name 526 xtra data: not used 527 528 reply 529 file position: not used 530 data: file stat info (format follows) 531 xtra data: not used 532 533 data format is the same as in directory listing with exception 534 that there is no file name appended. If file do not exists or 535 there is other problem (no access rights) return type of file is 536 0. 537 538 struct STAT { 539 long time; 540 long size; 541 byte type; 542 } 543 544 Compatibility 545 CC_ERR message is NEVER returned as reply to CC_STAT command 546 by server supporting CC_STAT command. If you have got CC_ERR reply, 547 you are talking to old server, which do not supports this 548 command. 549 550 CC_RENAME 0x4E - rename file or directory 551 request 552 file position: size of extra data 553 data: ASCIIZ source file or directory 554 xtra data: ASCIIZ destination file or directory 555 556 Note: It is possible to do cross-directory rename. In this 557 case you must have rights to DELETE in source directory and 558 to CREATE in target directory. 559 560 reply 561 file position: not used 562 data: not used 563 xtra data: not used 564 565 CC_CH_PASSW 0x4F - change password 566 not yet specified 567 568Reserved commands: 569 570 CC_LIMIT 0x80 - commands > 0x7F will have extended 571 header. No such extensions or commands 572 which uses that are known today. This 573 header will be used in protocol version 3. 574 575 CC_TEST 0x81 - reserved for testing of new header 576 577ignore this line vi: tw=70 578