10a6f33dbSBryan Gurneydm-dust 20a6f33dbSBryan Gurney======= 30a6f33dbSBryan Gurney 40a6f33dbSBryan GurneyThis target emulates the behavior of bad sectors at arbitrary 50a6f33dbSBryan Gurneylocations, and the ability to enable the emulation of the failures 60a6f33dbSBryan Gurneyat an arbitrary time. 70a6f33dbSBryan Gurney 80a6f33dbSBryan GurneyThis target behaves similarly to a linear target. At a given time, 90a6f33dbSBryan Gurneythe user can send a message to the target to start failing read 100a6f33dbSBryan Gurneyrequests on specific blocks (to emulate the behavior of a hard disk 110a6f33dbSBryan Gurneydrive with bad sectors). 120a6f33dbSBryan Gurney 130a6f33dbSBryan GurneyWhen the failure behavior is enabled (i.e.: when the output of 140a6f33dbSBryan Gurney"dmsetup status" displays "fail_read_on_bad_block"), reads of blocks 150a6f33dbSBryan Gurneyin the "bad block list" will fail with EIO ("Input/output error"). 160a6f33dbSBryan Gurney 170a6f33dbSBryan GurneyWrites of blocks in the "bad block list will result in the following: 180a6f33dbSBryan Gurney 190a6f33dbSBryan Gurney1. Remove the block from the "bad block list". 200a6f33dbSBryan Gurney2. Successfully complete the write. 210a6f33dbSBryan Gurney 220a6f33dbSBryan GurneyThis emulates the "remapped sector" behavior of a drive with bad 230a6f33dbSBryan Gurneysectors. 240a6f33dbSBryan Gurney 250a6f33dbSBryan GurneyNormally, a drive that is encountering bad sectors will most likely 260a6f33dbSBryan Gurneyencounter more bad sectors, at an unknown time or location. 270a6f33dbSBryan GurneyWith dm-dust, the user can use the "addbadblock" and "removebadblock" 280a6f33dbSBryan Gurneymessages to add arbitrary bad blocks at new locations, and the 290a6f33dbSBryan Gurney"enable" and "disable" messages to modulate the state of whether the 300a6f33dbSBryan Gurneyconfigured "bad blocks" will be treated as bad, or bypassed. 310a6f33dbSBryan GurneyThis allows the pre-writing of test data and metadata prior to 320a6f33dbSBryan Gurneysimulating a "failure" event where bad sectors start to appear. 330a6f33dbSBryan Gurney 340a6f33dbSBryan GurneyTable parameters 350a6f33dbSBryan Gurney---------------- 360a6f33dbSBryan Gurney<device_path> <offset> <blksz> 370a6f33dbSBryan Gurney 380a6f33dbSBryan GurneyMandatory parameters: 390a6f33dbSBryan Gurney <device_path>: 400a6f33dbSBryan Gurney Path to the block device. 410a6f33dbSBryan Gurney 420a6f33dbSBryan Gurney <offset>: 430a6f33dbSBryan Gurney Offset to data area from start of device_path 440a6f33dbSBryan Gurney 450a6f33dbSBryan Gurney <blksz>: 460a6f33dbSBryan Gurney Block size in bytes 470a6f33dbSBryan Gurney 480a6f33dbSBryan Gurney (minimum 512, maximum 1073741824, must be a power of 2) 490a6f33dbSBryan Gurney 500a6f33dbSBryan GurneyUsage instructions 510a6f33dbSBryan Gurney------------------ 520a6f33dbSBryan Gurney 530a6f33dbSBryan GurneyFirst, find the size (in 512-byte sectors) of the device to be used:: 540a6f33dbSBryan Gurney 550a6f33dbSBryan Gurney $ sudo blockdev --getsz /dev/vdb1 560a6f33dbSBryan Gurney 33552384 570a6f33dbSBryan Gurney 580a6f33dbSBryan GurneyCreate the dm-dust device: 590a6f33dbSBryan Gurney(For a device with a block size of 512 bytes) 600a6f33dbSBryan Gurney 610a6f33dbSBryan Gurney:: 620a6f33dbSBryan Gurney 630a6f33dbSBryan Gurney $ sudo dmsetup create dust1 --table '0 33552384 dust /dev/vdb1 0 512' 640a6f33dbSBryan Gurney 650a6f33dbSBryan Gurney(For a device with a block size of 4096 bytes) 660a6f33dbSBryan Gurney 670a6f33dbSBryan Gurney:: 680a6f33dbSBryan Gurney 690a6f33dbSBryan Gurney $ sudo dmsetup create dust1 --table '0 33552384 dust /dev/vdb1 0 4096' 700a6f33dbSBryan Gurney 710a6f33dbSBryan GurneyCheck the status of the read behavior ("bypass" indicates that all I/O 724f7f590bSyangerkunwill be passed through to the underlying device; "verbose" indicates that 734f7f590bSyangerkunbad block additions, removals, and remaps will be verbosely logged):: 740a6f33dbSBryan Gurney 750a6f33dbSBryan Gurney $ sudo dmsetup status dust1 764f7f590bSyangerkun 0 33552384 dust 252:17 bypass verbose 770a6f33dbSBryan Gurney 780a6f33dbSBryan Gurney $ sudo dd if=/dev/mapper/dust1 of=/dev/null bs=512 count=128 iflag=direct 790a6f33dbSBryan Gurney 128+0 records in 800a6f33dbSBryan Gurney 128+0 records out 810a6f33dbSBryan Gurney 820a6f33dbSBryan Gurney $ sudo dd if=/dev/zero of=/dev/mapper/dust1 bs=512 count=128 oflag=direct 830a6f33dbSBryan Gurney 128+0 records in 840a6f33dbSBryan Gurney 128+0 records out 850a6f33dbSBryan Gurney 860a6f33dbSBryan GurneyAdding and removing bad blocks 870a6f33dbSBryan Gurney------------------------------ 880a6f33dbSBryan Gurney 890a6f33dbSBryan GurneyAt any time (i.e.: whether the device has the "bad block" emulation 900a6f33dbSBryan Gurneyenabled or disabled), bad blocks may be added or removed from the 910a6f33dbSBryan Gurneydevice via the "addbadblock" and "removebadblock" messages:: 920a6f33dbSBryan Gurney 930a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 addbadblock 60 940a6f33dbSBryan Gurney kernel: device-mapper: dust: badblock added at block 60 950a6f33dbSBryan Gurney 960a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 addbadblock 67 970a6f33dbSBryan Gurney kernel: device-mapper: dust: badblock added at block 67 980a6f33dbSBryan Gurney 990a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 addbadblock 72 1000a6f33dbSBryan Gurney kernel: device-mapper: dust: badblock added at block 72 1010a6f33dbSBryan Gurney 1020a6f33dbSBryan GurneyThese bad blocks will be stored in the "bad block list". 1030a6f33dbSBryan GurneyWhile the device is in "bypass" mode, reads and writes will succeed:: 1040a6f33dbSBryan Gurney 1050a6f33dbSBryan Gurney $ sudo dmsetup status dust1 1060a6f33dbSBryan Gurney 0 33552384 dust 252:17 bypass 1070a6f33dbSBryan Gurney 1080a6f33dbSBryan GurneyEnabling block read failures 1090a6f33dbSBryan Gurney---------------------------- 1100a6f33dbSBryan Gurney 1110a6f33dbSBryan GurneyTo enable the "fail read on bad block" behavior, send the "enable" message:: 1120a6f33dbSBryan Gurney 1130a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 enable 1140a6f33dbSBryan Gurney kernel: device-mapper: dust: enabling read failures on bad sectors 1150a6f33dbSBryan Gurney 1160a6f33dbSBryan Gurney $ sudo dmsetup status dust1 1170a6f33dbSBryan Gurney 0 33552384 dust 252:17 fail_read_on_bad_block 1180a6f33dbSBryan Gurney 1190a6f33dbSBryan GurneyWith the device in "fail read on bad block" mode, attempting to read a 1200a6f33dbSBryan Gurneyblock will encounter an "Input/output error":: 1210a6f33dbSBryan Gurney 1220a6f33dbSBryan Gurney $ sudo dd if=/dev/mapper/dust1 of=/dev/null bs=512 count=1 skip=67 iflag=direct 1230a6f33dbSBryan Gurney dd: error reading '/dev/mapper/dust1': Input/output error 1240a6f33dbSBryan Gurney 0+0 records in 1250a6f33dbSBryan Gurney 0+0 records out 1260a6f33dbSBryan Gurney 0 bytes copied, 0.00040651 s, 0.0 kB/s 1270a6f33dbSBryan Gurney 1280a6f33dbSBryan Gurney...and writing to the bad blocks will remove the blocks from the list, 1290a6f33dbSBryan Gurneytherefore emulating the "remap" behavior of hard disk drives:: 1300a6f33dbSBryan Gurney 1310a6f33dbSBryan Gurney $ sudo dd if=/dev/zero of=/dev/mapper/dust1 bs=512 count=128 oflag=direct 1320a6f33dbSBryan Gurney 128+0 records in 1330a6f33dbSBryan Gurney 128+0 records out 1340a6f33dbSBryan Gurney 1350a6f33dbSBryan Gurney kernel: device-mapper: dust: block 60 removed from badblocklist by write 1360a6f33dbSBryan Gurney kernel: device-mapper: dust: block 67 removed from badblocklist by write 1370a6f33dbSBryan Gurney kernel: device-mapper: dust: block 72 removed from badblocklist by write 1380a6f33dbSBryan Gurney kernel: device-mapper: dust: block 87 removed from badblocklist by write 1390a6f33dbSBryan Gurney 1400a6f33dbSBryan GurneyBad block add/remove error handling 1410a6f33dbSBryan Gurney----------------------------------- 1420a6f33dbSBryan Gurney 1430a6f33dbSBryan GurneyAttempting to add a bad block that already exists in the list will 1440a6f33dbSBryan Gurneyresult in an "Invalid argument" error, as well as a helpful message:: 1450a6f33dbSBryan Gurney 1460a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 addbadblock 88 1470a6f33dbSBryan Gurney device-mapper: message ioctl on dust1 failed: Invalid argument 1480a6f33dbSBryan Gurney kernel: device-mapper: dust: block 88 already in badblocklist 1490a6f33dbSBryan Gurney 1500a6f33dbSBryan GurneyAttempting to remove a bad block that doesn't exist in the list will 1510a6f33dbSBryan Gurneyresult in an "Invalid argument" error, as well as a helpful message:: 1520a6f33dbSBryan Gurney 1530a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 removebadblock 87 1540a6f33dbSBryan Gurney device-mapper: message ioctl on dust1 failed: Invalid argument 1550a6f33dbSBryan Gurney kernel: device-mapper: dust: block 87 not found in badblocklist 1560a6f33dbSBryan Gurney 1570a6f33dbSBryan GurneyCounting the number of bad blocks in the bad block list 1580a6f33dbSBryan Gurney------------------------------------------------------- 1590a6f33dbSBryan Gurney 1600a6f33dbSBryan GurneyTo count the number of bad blocks configured in the device, run the 1610a6f33dbSBryan Gurneyfollowing message command:: 1620a6f33dbSBryan Gurney 1630a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 countbadblocks 1640a6f33dbSBryan Gurney 1650a6f33dbSBryan GurneyA message will print with the number of bad blocks currently 1660a6f33dbSBryan Gurneyconfigured on the device:: 1670a6f33dbSBryan Gurney 1684f7f590bSyangerkun countbadblocks: 895 badblock(s) found 1690a6f33dbSBryan Gurney 1700a6f33dbSBryan GurneyQuerying for specific bad blocks 1710a6f33dbSBryan Gurney-------------------------------- 1720a6f33dbSBryan Gurney 1730a6f33dbSBryan GurneyTo find out if a specific block is in the bad block list, run the 1740a6f33dbSBryan Gurneyfollowing message command:: 1750a6f33dbSBryan Gurney 1760a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 queryblock 72 1770a6f33dbSBryan Gurney 1780a6f33dbSBryan GurneyThe following message will print if the block is in the list:: 1790a6f33dbSBryan Gurney 1804f7f590bSyangerkun dust_query_block: block 72 found in badblocklist 1810a6f33dbSBryan Gurney 1820a6f33dbSBryan GurneyThe following message will print if the block is not in the list:: 1830a6f33dbSBryan Gurney 1844f7f590bSyangerkun dust_query_block: block 72 not found in badblocklist 1850a6f33dbSBryan Gurney 1860a6f33dbSBryan GurneyThe "queryblock" message command will work in both the "enabled" 1870a6f33dbSBryan Gurneyand "disabled" modes, allowing the verification of whether a block 1880a6f33dbSBryan Gurneywill be treated as "bad" without having to issue I/O to the device, 1890a6f33dbSBryan Gurneyor having to "enable" the bad block emulation. 1900a6f33dbSBryan Gurney 1910a6f33dbSBryan GurneyClearing the bad block list 1920a6f33dbSBryan Gurney--------------------------- 1930a6f33dbSBryan Gurney 1940a6f33dbSBryan GurneyTo clear the bad block list (without needing to individually run 1950a6f33dbSBryan Gurneya "removebadblock" message command for every block), run the 1960a6f33dbSBryan Gurneyfollowing message command:: 1970a6f33dbSBryan Gurney 1980a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 clearbadblocks 1990a6f33dbSBryan Gurney 2000a6f33dbSBryan GurneyAfter clearing the bad block list, the following message will appear:: 2010a6f33dbSBryan Gurney 2024f7f590bSyangerkun dust_clear_badblocks: badblocks cleared 2030a6f33dbSBryan Gurney 2040a6f33dbSBryan GurneyIf there were no bad blocks to clear, the following message will 2050a6f33dbSBryan Gurneyappear:: 2060a6f33dbSBryan Gurney 2074f7f590bSyangerkun dust_clear_badblocks: no badblocks found 2080a6f33dbSBryan Gurney 209*0c248ea2SyangerkunListing the bad block list 210*0c248ea2Syangerkun-------------------------- 211*0c248ea2Syangerkun 212*0c248ea2SyangerkunTo list all bad blocks in the bad block list (using an example device 213*0c248ea2Syangerkunwith blocks 1 and 2 in the bad block list), run the following message 214*0c248ea2Syangerkuncommand:: 215*0c248ea2Syangerkun 216*0c248ea2Syangerkun $ sudo dmsetup message dust1 0 listbadblocks 217*0c248ea2Syangerkun 1 218*0c248ea2Syangerkun 2 219*0c248ea2Syangerkun 220*0c248ea2SyangerkunIf there are no bad blocks in the bad block list, the command will 221*0c248ea2Syangerkunexecute with no output:: 222*0c248ea2Syangerkun 223*0c248ea2Syangerkun $ sudo dmsetup message dust1 0 listbadblocks 224*0c248ea2Syangerkun 2250a6f33dbSBryan GurneyMessage commands list 2260a6f33dbSBryan Gurney--------------------- 2270a6f33dbSBryan Gurney 2280a6f33dbSBryan GurneyBelow is a list of the messages that can be sent to a dust device: 2290a6f33dbSBryan Gurney 2300a6f33dbSBryan GurneyOperations on blocks (requires a <blknum> argument):: 2310a6f33dbSBryan Gurney 2320a6f33dbSBryan Gurney addbadblock <blknum> 2330a6f33dbSBryan Gurney queryblock <blknum> 2340a6f33dbSBryan Gurney removebadblock <blknum> 2350a6f33dbSBryan Gurney 2360a6f33dbSBryan Gurney...where <blknum> is a block number within range of the device 2370a6f33dbSBryan Gurney(corresponding to the block size of the device.) 2380a6f33dbSBryan Gurney 2390a6f33dbSBryan GurneySingle argument message commands:: 2400a6f33dbSBryan Gurney 2410a6f33dbSBryan Gurney countbadblocks 2420a6f33dbSBryan Gurney clearbadblocks 243*0c248ea2Syangerkun listbadblocks 2440a6f33dbSBryan Gurney disable 2450a6f33dbSBryan Gurney enable 2460a6f33dbSBryan Gurney quiet 2470a6f33dbSBryan Gurney 2480a6f33dbSBryan GurneyDevice removal 2490a6f33dbSBryan Gurney-------------- 2500a6f33dbSBryan Gurney 2510a6f33dbSBryan GurneyWhen finished, remove the device via the "dmsetup remove" command:: 2520a6f33dbSBryan Gurney 2530a6f33dbSBryan Gurney $ sudo dmsetup remove dust1 2540a6f33dbSBryan Gurney 2550a6f33dbSBryan GurneyQuiet mode 2560a6f33dbSBryan Gurney---------- 2570a6f33dbSBryan Gurney 2580a6f33dbSBryan GurneyOn test runs with many bad blocks, it may be desirable to avoid 2590a6f33dbSBryan Gurneyexcessive logging (from bad blocks added, removed, or "remapped"). 2600a6f33dbSBryan GurneyThis can be done by enabling "quiet mode" via the following message:: 2610a6f33dbSBryan Gurney 2620a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 quiet 2630a6f33dbSBryan Gurney 2640a6f33dbSBryan GurneyThis will suppress log messages from add / remove / removed by write 2650a6f33dbSBryan Gurneyoperations. Log messages from "countbadblocks" or "queryblock" 2660a6f33dbSBryan Gurneymessage commands will still print in quiet mode. 2670a6f33dbSBryan Gurney 2680a6f33dbSBryan GurneyThe status of quiet mode can be seen by running "dmsetup status":: 2690a6f33dbSBryan Gurney 2700a6f33dbSBryan Gurney $ sudo dmsetup status dust1 2710a6f33dbSBryan Gurney 0 33552384 dust 252:17 fail_read_on_bad_block quiet 2720a6f33dbSBryan Gurney 2730a6f33dbSBryan GurneyTo disable quiet mode, send the "quiet" message again:: 2740a6f33dbSBryan Gurney 2750a6f33dbSBryan Gurney $ sudo dmsetup message dust1 0 quiet 2760a6f33dbSBryan Gurney 2770a6f33dbSBryan Gurney $ sudo dmsetup status dust1 2780a6f33dbSBryan Gurney 0 33552384 dust 252:17 fail_read_on_bad_block verbose 2790a6f33dbSBryan Gurney 2800a6f33dbSBryan Gurney(The presence of "verbose" indicates normal logging.) 2810a6f33dbSBryan Gurney 2820a6f33dbSBryan Gurney"Why not...?" 2830a6f33dbSBryan Gurney------------- 2840a6f33dbSBryan Gurney 2850a6f33dbSBryan Gurneyscsi_debug has a "medium error" mode that can fail reads on one 2860a6f33dbSBryan Gurneyspecified sector (sector 0x1234, hardcoded in the source code), but 2870a6f33dbSBryan Gurneyit uses RAM for the persistent storage, which drastically decreases 2880a6f33dbSBryan Gurneythe potential device size. 2890a6f33dbSBryan Gurney 2900a6f33dbSBryan Gurneydm-flakey fails all I/O from all block locations at a specified time 2910a6f33dbSBryan Gurneyfrequency, and not a given point in time. 2920a6f33dbSBryan Gurney 2930a6f33dbSBryan GurneyWhen a bad sector occurs on a hard disk drive, reads to that sector 2940a6f33dbSBryan Gurneyare failed by the device, usually resulting in an error code of EIO 2950a6f33dbSBryan Gurney("I/O error") or ENODATA ("No data available"). However, a write to 2960a6f33dbSBryan Gurneythe sector may succeed, and result in the sector becoming readable 2970a6f33dbSBryan Gurneyafter the device controller no longer experiences errors reading the 2980a6f33dbSBryan Gurneysector (or after a reallocation of the sector). However, there may 2990a6f33dbSBryan Gurneybe bad sectors that occur on the device in the future, in a different, 3000a6f33dbSBryan Gurneyunpredictable location. 3010a6f33dbSBryan Gurney 3020a6f33dbSBryan GurneyThis target seeks to provide a device that can exhibit the behavior 3030a6f33dbSBryan Gurneyof a bad sector at a known sector location, at a known time, based 3040a6f33dbSBryan Gurneyon a large storage device (at least tens of gigabytes, not occupying 3050a6f33dbSBryan Gurneysystem memory). 306