1Safe copying of files and partitions. 2 3Idea: 4 5A main problem with damaged storage hardware is, that once you get an 6unrecoverable IO error further reading from the file / device often fails until 7the file has been closed and re-opened. 8 9The normal copy tools like cat, cp or dd do not allow creation of an image file 10from a disk or CD-ROM once reading of a sector failed. 11 12Safecopy tries to get as much data from the source as possible, even resorting 13to device specific low level operations if applicable. 14 15This is achieved by identifying problematic or damaged areas, skipping over 16them and continuing reading afterwards. The corresponding area in the 17destination file is either skipped (on initial creation that means padded with 18zeros) or deliberately filled with a recognizable pattern to later find 19affected files on a corrupted device. 20Safecopy uses an incremental algorithm to identify the exact beginning and end 21of bad areas, allowing the user to trade minimum accesses to bad areas for 22thorough data resurrection. 23Multiple passes over the same file are possible, to first retrieve as much data 24from a device as possible with minimum harm, and then trying to retrieve some 25of the remaining data with increasingly aggressive read attempts. 26 27For this to work, the source device or file has to be seekable. For unseekable 28devices (like tapes) you can try to use an external script to execute a 29controlled skip over the damaged part for you. 30(For example by using "mt seek" and "mt tell" on an SCSI tape device) 31See the "-S <seekscript>" parameter for details. 32 33Performance and success of this tool depend extremely on the device driver, 34firmware and underlying hardware. 35 36Currently safecopy supports RAW access to CDROM drives to read data directly of 37a CD, bypassing some driver dependant error correction. This can speed up data 38retrieval from CDs and reduce system load during recovery, as well as increase 39the success rate. Safecopy uses the disc status syscall to determine sector 40size and addressing of CDs. This fails on mixed-mode or multi-session CDs, 41since the sector layout can change within the disk, but would still work on the 42the big majority of disks. Other disks can still be recovered using normal high 43level data access. Safecopy auto-detects the disk type involved during the scan 44for disk and block size. 45 46Some CD/DVD drives are known to cause the ATAPI bus to crash on errors, causing 47the device driver to freeze for times up to and beyond a minute per error. Try 48to avoid using such drives for media recovery. Using safecopys low level access 49features might help under some circumstances. 50 51Some drives can read bad media better than others. Be sure to attempt data 52recovery of CDs and DVDs on several different drives and computers. 53You can use safecopys incremental recovery feature to read previously 54unreadable sectors only. 55 56 57Reliability: 58 59Data recovery from damaged media is a delicate task, in the worst case its 60success or failure can safe or ruin whole companies and seal affected peoples 61personal fate. It is paramount that any tools written for that purpose are 62reliable and trustworthy. 63 64A user needs to know what exactly the software is doing to his hardware and 65data. The outcome of any operation needs to be both understandable and 66predictable. 67 68An "intelligent data resurrection wizard" with unknown complex internal 69behaviour may be a nifty tool, but does not meet the above requirements of 70predictable outcome, nor will the user know in advance what is done to his 71data. 72 73The operation sequence of safecopy has been kept relatively simple to assure 74this predictability. Unfortunately feature additions have risen the complexity 75and lead to undefined outcome in the past when include and exclude lists had 76been mixed, especially when mixing different block sizes. In the worst case 77this could have lead to overwritten data in the destination file on a later 78incremental run with the mark (-M) option. 79 80From version 1.3 on, safecopy ships with a test suite that can be used to verify 81safecopys behaviour in a set of test cases, simulating the combination of bad 82blocks in input with different include and exclude lists, both with and without 83marking. Releases are only made if safecopy passes those test cases according 84to the specification. 85 86This textual specification of behaviour of safecopy can be found in the file 87specification.txt shipped with safecopy. 88 89 90Different use cases: 91 92 How do I... 93 94- resurrect a file from a mounted but damaged media, that copy will fail on: 95 safecopy /path/to/problemfile ~/saved-file 96 97- create an filesystem image of a damaged disk/cdrom: 98 safecopy /dev/device ~/diskimage 99 100- resurrect data as thoroughly as possible? 101 safecopy source dest -f 1* -R 8 -Z 2 102 (assuming logical misalignment of blocks to sectors) 103 safecopy source dest -f 1* -r 1 -R 8 -Z 2 104 105- resurrect data as fast as possible, or 106- resurrect data with low risk of damaging the media further: 107 safecopy source dest -f 10% -R 0 -Z 0 108 109- resurrect some data fast, then read more data thoroughly later: 110 safecopy source dest -f 10% -R 0 -Z 0 -o badblockfile 111 safecopy source dest -f 1* -R 8 -Z 2 -I badblockfile 112 Alternate approach using the new preset features: 113 safecopy source dest --stage1 114 safecopy source dest --stage2 115 safecopy source dest --stage3 116 117- utilize some friends CD-ROM drives to complete the data from my damaged CD: 118 safecopy /dev/mydrive imagefile <someoptions> -b <myblocksize> \ 119 -o myblockfile; 120 safecopy /dev/otherdrive imagefile <someoptions> -b <otherblocksize> \ 121 -I myblockfile -i <myblocksize> -o otherblockfile; 122 safecopy /dev/anotherdrive imagefile <someoptions> \ 123 -b <anotherblocksize> -I otherblockfile -i <otherblocksize> 124 125- interrupt and later resume a data rescue operation: 126 safecopy source dest 127 <CTRL+C> (safecopy aborts) 128 safecopy source dest -I /dev/null 129 130- interrupt and later resume a data rescue operation with correct 131 badblocks output: 132 safecopy source dest <options> -o badblockfile 133 <CTRL+C> (safecopy aborts) 134 mv badblockfile savedbadblockfile 135 safecopy source dest -I /dev/null -o badblockfile 136 cat badblockfile >>savedbadblockfile 137 138- interrupt and resume in incremental mode: 139 (this needs a bit of bash scripting to get the correct badblock lists) 140 safecopy source dest <options> -o badblockfile1 141 safecopy source dest <options> -I badblockfile1 -o badblockfile2 142 <CTRL+C> (safecopy aborts) 143 latest=$( tail -n 1 badblockfile2 ) 144 if [ -z $latest ]; then latest=-1; fi; 145 cat badblockfile1 | while read block; do 146 [ $block -gt $latest ] && echo $block >>badblockfile2; 147 done; 148 safecopy source dest <options> -I badblockfile2 -o badblockfile3 149 150- find the corrupted files on a partially successful rescued file system: 151 safecopy /dev/filesystem image -M CoRrUpTeD 152 fsck image 153 mount -o loop image /mnt/mountpoint 154 grep -R /mnt/mountpoint "CoRrUpTeD" 155 (hint: this might not find all affected files if the unreadable 156 parts are smaller in size than your marker string) 157 158- exclude the previously known badblocks list of a filesystem from 159 filesystem image creation: 160 dumpe2fs -b /dev/filesystem >badblocklist 161 safecopy /dev/filesystem image \ 162 -X badblocklist -x <blocksize of your fs> 163 164- create an image of a device that starts at X and is Y in size: 165 safecopy /dev/filesystem -b <bsize> -s <X/bsize> -l <Y/bsize> 166 167- combine two partial images of rescued data without access to the actual 168 (damaged) source data: 169 (This is a bit tricky. You need to get badblocks lists for both files 170 somehow to make safecopy know where the missing data is. If you used 171 the -M (mark) feature you might be able to automatically compute these, 172 however this feature is not provided by safecopy. Lets assume you have 173 two badblocks files. 174 you have: image1.dat 175 image1.badblocks (blocksize1) 176 image2.dat 177 image2.badblocks (blocksize2) 178 The file size of image1 needs to be greater or equal to that of image2. 179 (If not, swap them) 180 ) 181 182 cp image2.dat combined.dat 183 safecopy image1.dat combined.dat -I image2.badblocks -i blocksize2 \ 184 -X image1.badblocks -x blocksize1 185 186 (This gets you the combined data, but no output badblocklist. 187 The resulting badblocks list would be the badblocks that are 188 a: in both badblocks lists, or 189 b: in image1.badblocks and beyond the file size of image2 190 It should be reasonably easy to solve this logic in a short shell 191 script. One day this might be shipped with safecopy, until then 192 consider this your chance to contribute to a random open source 193 project.) 194 195- rescue data of a tape device: 196 If the tape device driver supports lseek(), treat it as any file, 197 otherwise utilize the "-S" option of safecopy with a to be 198 self-written script to skip over the bad blocks. 199 (for example using "mt seek") 200 Make sure your tape device doesn't auto-rewind on close. 201 Send me feedback if you had any luck doing so, so I can update 202 this documentation. 203 204 205FAQ: 206 Q: Why create this tool if there already is something like dd-rescue and 207 other tools for that purpose? 208 A: Because I didn't know of dd(-)rescue when I started, and I felt like 209 it. Also I think safecopy suits the needs of a user in data loss peril 210 better due to more readable output and more understandable options than 211 some of the other tools. (Then again I am biased. Compare them 212 yourself.) Meanwhile safecopy supports low level features other tools 213 don't. 214 215 Q: What exactly does the -Z option do? 216 A: Remember back in MS-DOS times when a floppy would make a "neek nark" 217 sound 3 times every time when running into a read error? 218 This happened when the BIOS or DOS disk driver moved the IO head 219 to its boundaries to possibly correct small cylinder misalignment, 220 before it tried again. 221 Linux doesn't do that by default, neither do common CDROM drives or 222 drivers. Nevertheless forcing this behaviour can increase your 223 chance of reading bad sectors from a CD __BIG__ time. 224 (Unlike floppies where it usually has little effect) 225 226 Q: Whats my best chance to resurrect a CD that has become unreadable? 227 A: Try making a backup image on many different computers and drives. 228 The abilities to read from bad media vary extremely. I have a 6 year 229 old Lite On CDRW drive, that even reads deeply and purposely scratched 230 CDs (as in with my key, to make it unreadable) flawlessly. A CDRW drive 231 of the same age at work doesn't read any data from that part of the CD 232 at all, while most DVD and combo drives have bad blocks every couple 233 hundred bytes. 234 Make full use of safecopys RAW access features if applicable. 235 (-L 2 option) 236 As a general guideline: 237 -CDRW drives usually do better than read-only CD drives. 238 -CD only drives sometimes do better on CDs than DVD drives. 239 -PC drives are sometimes better than laptop ones. 240 -A drive with a clean lens does better than a dirtball. 241 -Cleaning up CDs helps. 242 -Unless you use chemicals. 243 -Using sticky tape on a CD will rip of the reflective layer 244 permanently rendering the disk unreadable. 245 246 Q: Whats my best chance to resurrect a floppy that became unreadable? 247 A: Again try different floppy drives. Keep in mind that it might be 248 easier to further damage data on a bad floppy than on a CD. 249 (Don't overdo read attempts) 250 251 Q: What about BlueRay/HDDVD disks? 252 A: Hell if I knew, but generally they should be similar to DVDs. 253 It probably depends how the drives firmware acts up. 254 255 Q: My hard drive suddenly has many bad sectors, what should I do? 256 A: Speed is an essential factor when rescuing data from a bad hard drive. 257 Accesses to bad areas or even just time running can further damage the 258 drive and make formerly readable areas unreadable, be it due to 259 temperature rise, damaged heads scratching still good parts of the 260 surface, bearings degrading due to vibration, etc. Its advisable to 261 shut the system down and remove the hard drive from the computer as soon 262 as errors occur and as fast as possible without causing further damage. 263 (Don't pull the plug! Press reset to force reboot and then power down 264 via power button/ACPI) 265 Set up a rescue system with enough disk space to store all the data 266 from the damaged drive (and possibly multiple copies of it). If you 267 have an external hard drive case that connects via USB SATA or SCSI, 268 allowing a hot plug of the drive into a running system, use it. This 269 allows you to prepare everything without the need for the drive to power 270 up and possibly BIOS or operating system involuntarily accessing it. 271 You also get easier access to the drive to check temperature or noise 272 during operation. 273 When you rescue data, rescue good data first before attempting to 274 access bad sectors. Safecopy allows you to skip known problem sectors 275 using a badblock exclude file (-X) which you might be able to retrieve 276 from system logs or from the drive internal logs, via smartmontools or 277 similar software. Be aware that you might possibly need to convert 278 physical sector numbers into logical block numbers depending on your 279 source. 280 Also you should tell safecopy to jump out of any problematic areas ASAP 281 and continue somewhere else. Parameters "-f 10% -r 10% -R 0 -Z 0" would 282 do the trick by making safecopy skip 10% of the device content and 283 continue there without backtracking. You can always attempt to get the 284 data in between later, first get the supposedly good data on the rest 285 of the drive. Depending on the method of data recovery you plan on 286 using, it may make sense to mark the bad data with the "-M" option. 287 This allows you to later find files affected by data corruption more 288 easily. Use the "-o" option to make safecopy write a badblock list 289 with all blocks skipped or unreadable. 290 When safecopy is done with this first run, you can attempt a second go 291 trying to get more data. Using smaller values for "-f" and allowing 292 safecopy to backtrack for the end of the affected area "-r 1*". 293 Be sure to use incremental mode "-I" to only read the blocks skipped 294 in the first run. 295 It may make sense to let the drive cool down between runs. 296 Once you got all the data from the "good" areas of the drive you can 297 risk more "thorough" access to it. Increase the numbers of retries 298 "-R" in case of bad blocks, maybe together with a head realignment 299 "-Z 1". "-f 1*" would make safecopy try to read on every single block, 300 not skipping over bad areas at all. 301 If your drive stops responding, power it down, let it cool down for a 302 while, then try again. 303 (I heard from people who brought dead drives back to live for a short 304 time by cooling them to low temperatures with ice-packs.) 305 306 !!! If the data is really important, go to a professional data recovery 307 !!! specialist right away, before doing further damage to the drive. 308 309 310Safecopy 1.7 by CorvusCorax 311Usage: safecopy [options] <source> <target> 312Options: 313 --stage1 : Preset to rescue most of the data fast, 314 using no retries and avoiding bad areas. 315 Presets: -f 10% -r 10% -R 1 -Z 0 -L 2 -M BaDbLoCk 316 -o stage1.badblocks 317 --stage2 : Preset to rescue more data, using no retries 318 but searching for exact ends of bad areas. 319 Presets: -f 128* -r 1* -R 1 -Z 0 -L 2 320 -I stage1.badblocks 321 -o stage2.badblocks 322 --stage3 : Preset to rescue everything that can be rescued 323 using maximum retries, head realignment tricks 324 and low level access. 325 Presets: -f 1* -r 1* -R 4 -Z 1 -L 2 326 -I stage2.badblocks 327 -o stage3.badblocks 328 All stage presets can be overridden by individual options. 329 -b <size> : Blocksize for default read operations. 330 Set this to the physical sectorsize of your media. 331 Default: 1* 332 Hardware block size if reported by OS, otherwise 4096 333 -f <size> : Blocksize when skipping over badblocks. 334 Higher settings put less strain on your hardware, 335 but you might miss good areas in between two bad ones. 336 Default: 16* 337 -r <size> : Resolution in bytes when searching for the exact 338 beginning or end of a bad area. 339 If you read data directly from a device there is no 340 need to set this lower than the hardware blocksize. 341 On mounted filesystems however, read blocks 342 and physical blocks could be misaligned. 343 Smaller values lead to very thorough attempts to read 344 data at the edge of damaged areas, 345 but increase the strain on the damaged media. 346 Default: 1* 347 -R <number> : At least that many read attempts are made on the first 348 bad block of a damaged area with minimum resolution. 349 More retries can sometimes recover a weak sector, 350 but at the cost of additional strain. 351 Default: 3 352 -Z <number> : On each error, force seek the read head from start to 353 end of the source device as often as specified. 354 That takes time, creates additional strain and might 355 not be supported by all devices or drivers. 356 Default: 1 357 -L <mode> : Use low level device calls as specified: 358 0 Do not use low level device calls 359 1 Attempt low level device calls 360 for error recovery only 361 2 Always use low level device calls 362 if available 363 Supported low level features in this version are: 364 SYSTEM DEVICE TYPE FEATURE 365 Linux cdrom/dvd bus/device reset 366 Linux cdrom read sector in raw mode 367 Linux floppy controller reset, twaddle 368 Default: 1 369 --sync : Use synchronized read calls (disable driver buffering). 370 Safecopy will use O_DIRECT if supported by the OS 371 and O_SYNC otherwise. 372 Default: Asynchronous read buffering by the OS is allowed 373 --forceopen : Keep trying to reopen the source after a read errer 374 useful for USB drives that go away temporarily. 375 Warning: This can cause safecopy to hang 376 until aborted manually! 377 Default: Abort on fopen() error 378 -s <blocks> : Start position where to start reading. 379 Will correspond to position 0 in the destination file. 380 Default: block 0 381 -l <blocks> : Maximum length of data to be read. 382 Default: Entire size of input file 383 -I <badblockfile> : Incremental mode. Assume the target file already 384 exists and has holes specified in the badblockfile. 385 It will be attempted to retrieve more data from 386 the listed blocks or from beyond the file size 387 of the target file only. 388 Warning: Without this option, the destination file 389 will be emptied prior to writing. 390 Use -I /dev/null if you want to continue a previous 391 run of safecopy without a badblock list. 392 Implies: -c 0 if -c is not specified 393 Default: none ( /dev/null if -c is given ) 394 -i <bytes> : Blocksize to interpret the badblockfile given with -I. 395 Default: Blocksize as specified by -b 396 -c <blocks> : Continue copying at this position. 397 This allows continuing if the output is a block device 398 with a fixed size as opposed to a growable file, 399 where safecopy cannot determine how far it already got. 400 The blocksize used is the same as for the -I option. 401 -c 0 will continue at the current destination size. 402 Implies: -I /dev/null if -I is not specified 403 Default: none, 0 if -I is specified 404 -X <badblockfile> : Exclusion mode. If used together with -I, 405 excluded blocks override included blocks. 406 Safecopy will not read or write any data from 407 areas covered by exclude blocks. 408 Default: none ( 0 if -I is given ) 409 -x <bytes> : Blocksize to interpret the badblockfile given with -X. 410 Default: Blocksize as specified by -b 411 -o <badblockfile> : Write a badblocks/e2fsck compatible bad block file. 412 Default: none 413 -S <seekscript> : Use external script for seeking in input file. 414 (Might be useful for tape devices and similar). 415 Seekscript must be an executable that takes the 416 number of blocks to be skipped as argv1 (1-64) 417 the blocksize in bytes as argv2 418 and the current position (in bytes) as argv3. 419 Return value needs to be the number of blocks 420 successfully skipped, or 0 to indicate seek failure. 421 The external seekscript will only be used 422 if lseek() fails and we need to skip over data. 423 Default: none 424 -M <string> : Mark unrecovered data with this string instead of 425 skipping it. This helps in later finding corrupted 426 files on rescued file system images. 427 The default is to zero unreadable data on creation 428 of output files, and leaving the data as it is 429 on any later run. 430 Warning: When used in combination with 431 incremental mode (-I) this may overwrite data 432 in any block that occurs in the -I file. 433 Blocks not in the -I file, or covered by the file 434 specified with -X are save from being overwritten. 435 Default: none 436 --debug <level> : Enable debug output. Level is a bit field, 437 add values together for more information: 438 program flow: 1 439 IO control: 2 440 badblock marking: 4 441 seeking: 8 442 incremental mode: 16 443 exclude mode: 32 444 or for all debug output: 255 445 Default: 0 446 -T <timingfile> : Write sector read timing information into 447 this file for later analysis. 448 Default: none 449 -h | --help : Show this text 450 451Valid parameters for -f -r -b <size> options are: 452 <integer> Amount in bytes - i.e. 1024 453 <percentage>% Percentage of whole file/device size - e.g. 10% 454 <number>* -b only, number times blocksize reported by OS 455 <number>* -f and -r only, number times the value of -b 456 457Description of output: 458 . : Between 1 and 1024 blocks successfully read. 459 _ : Read of block was incomplete. (possibly end of file) 460 The blocksize is now reduced to read the rest. 461 |/| : Seek failed, source can only be read sequentially. 462 > : Read failed, reducing blocksize to read partial data. 463 ! : A low level error on read attempt of smallest allowed size 464 leads to a retry attempt. 465 [xx](+yy){ : Current block and number of bytes continuously 466 read successfully up to this point. 467 X : Read failed on a block with minimum blocksize and is skipped. 468 Unrecoverable error, destination file is padded with zeros. 469 Data is now skipped until end of the unreadable area is reached. 470 < : Successful read after the end of a bad area causes 471 backtracking with smaller blocksizes to search for the first 472 readable data. 473 }[xx](+yy) : current block and number of bytes of recent 474 continuous unreadable data. 475 476Copyright 2012 CorvusCorax 477This is free software. You may redistribute copies of it under 478the terms of the GNU General Public License version 2 or above. 479 <http://www.gnu.org/licenses/gpl2.html>. 480There is NO WARRANTY, to the extent permitted by law. 481