104bf7ac6SMatthew Sakai.. SPDX-License-Identifier: GPL-2.0-only 204bf7ac6SMatthew Sakai 304bf7ac6SMatthew Sakaidm-vdo 404bf7ac6SMatthew Sakai====== 504bf7ac6SMatthew Sakai 604bf7ac6SMatthew SakaiThe dm-vdo (virtual data optimizer) device mapper target provides 704bf7ac6SMatthew Sakaiblock-level deduplication, compression, and thin provisioning. As a device 804bf7ac6SMatthew Sakaimapper target, it can add these features to the storage stack, compatible 904bf7ac6SMatthew Sakaiwith any file system. The vdo target does not protect against data 1004bf7ac6SMatthew Sakaicorruption, relying instead on integrity protection of the storage below 1104bf7ac6SMatthew Sakaiit. It is strongly recommended that lvm be used to manage vdo volumes. See 1204bf7ac6SMatthew Sakailvmvdo(7). 1304bf7ac6SMatthew Sakai 1404bf7ac6SMatthew SakaiUserspace component 1504bf7ac6SMatthew Sakai=================== 1604bf7ac6SMatthew Sakai 1704bf7ac6SMatthew SakaiFormatting a vdo volume requires the use of the 'vdoformat' tool, available 1804bf7ac6SMatthew Sakaiat: 1904bf7ac6SMatthew Sakai 2004bf7ac6SMatthew Sakaihttps://github.com/dm-vdo/vdo/ 2104bf7ac6SMatthew Sakai 2204bf7ac6SMatthew SakaiIn most cases, a vdo target will recover from a crash automatically the 2304bf7ac6SMatthew Sakainext time it is started. In cases where it encountered an unrecoverable 2404bf7ac6SMatthew Sakaierror (either during normal operation or crash recovery) the target will 2504bf7ac6SMatthew Sakaienter or come up in read-only mode. Because read-only mode is indicative of 2604bf7ac6SMatthew Sakaidata-loss, a positive action must be taken to bring vdo out of read-only 2704bf7ac6SMatthew Sakaimode. The 'vdoforcerebuild' tool, available from the same repo, is used to 2804bf7ac6SMatthew Sakaiprepare a read-only vdo to exit read-only mode. After running this tool, 2904bf7ac6SMatthew Sakaithe vdo target will rebuild its metadata the next time it is 3004bf7ac6SMatthew Sakaistarted. Although some data may be lost, the rebuilt vdo's metadata will be 3104bf7ac6SMatthew Sakaiinternally consistent and the target will be writable again. 3204bf7ac6SMatthew Sakai 3304bf7ac6SMatthew SakaiThe repo also contains additional userspace tools which can be used to 3404bf7ac6SMatthew Sakaiinspect a vdo target's on-disk metadata. Fortunately, these tools are 3504bf7ac6SMatthew Sakairarely needed except by dm-vdo developers. 3604bf7ac6SMatthew Sakai 37*cb824724SMatthew SakaiMetadata requirements 38*cb824724SMatthew Sakai===================== 39*cb824724SMatthew Sakai 40*cb824724SMatthew SakaiEach vdo volume reserves 3GB of space for metadata, or more depending on 41*cb824724SMatthew Sakaiits configuration. It is helpful to check that the space saved by 42*cb824724SMatthew Sakaideduplication and compression is not cancelled out by the metadata 43*cb824724SMatthew Sakairequirements. An estimation of the space saved for a specific dataset can 44*cb824724SMatthew Sakaibe computed with the vdo estimator tool, which is available at: 45*cb824724SMatthew Sakai 46*cb824724SMatthew Sakaihttps://github.com/dm-vdo/vdoestimator/ 47*cb824724SMatthew Sakai 4804bf7ac6SMatthew SakaiTarget interface 4904bf7ac6SMatthew Sakai================ 5004bf7ac6SMatthew Sakai 5104bf7ac6SMatthew SakaiTable line 5204bf7ac6SMatthew Sakai---------- 5304bf7ac6SMatthew Sakai 5404bf7ac6SMatthew Sakai:: 5504bf7ac6SMatthew Sakai 5604bf7ac6SMatthew Sakai <offset> <logical device size> vdo V4 <storage device> 5704bf7ac6SMatthew Sakai <storage device size> <minimum I/O size> <block map cache size> 5804bf7ac6SMatthew Sakai <block map era length> [optional arguments] 5904bf7ac6SMatthew Sakai 6004bf7ac6SMatthew Sakai 6104bf7ac6SMatthew SakaiRequired parameters: 6204bf7ac6SMatthew Sakai 6304bf7ac6SMatthew Sakai offset: 6404bf7ac6SMatthew Sakai The offset, in sectors, at which the vdo volume's logical 6504bf7ac6SMatthew Sakai space begins. 6604bf7ac6SMatthew Sakai 6704bf7ac6SMatthew Sakai logical device size: 6804bf7ac6SMatthew Sakai The size of the device which the vdo volume will service, 6904bf7ac6SMatthew Sakai in sectors. Must match the current logical size of the vdo 7004bf7ac6SMatthew Sakai volume. 7104bf7ac6SMatthew Sakai 7204bf7ac6SMatthew Sakai storage device: 7304bf7ac6SMatthew Sakai The device holding the vdo volume's data and metadata. 7404bf7ac6SMatthew Sakai 7504bf7ac6SMatthew Sakai storage device size: 7604bf7ac6SMatthew Sakai The size of the device holding the vdo volume, as a number 7704bf7ac6SMatthew Sakai of 4096-byte blocks. Must match the current size of the vdo 7804bf7ac6SMatthew Sakai volume. 7904bf7ac6SMatthew Sakai 8004bf7ac6SMatthew Sakai minimum I/O size: 8104bf7ac6SMatthew Sakai The minimum I/O size for this vdo volume to accept, in 8204bf7ac6SMatthew Sakai bytes. Valid values are 512 or 4096. The recommended value 8304bf7ac6SMatthew Sakai is 4096. 8404bf7ac6SMatthew Sakai 8504bf7ac6SMatthew Sakai block map cache size: 8604bf7ac6SMatthew Sakai The size of the block map cache, as a number of 4096-byte 8704bf7ac6SMatthew Sakai blocks. The minimum and recommended value is 32768 blocks. 8804bf7ac6SMatthew Sakai If the logical thread count is non-zero, the cache size 8904bf7ac6SMatthew Sakai must be at least 4096 blocks per logical thread. 9004bf7ac6SMatthew Sakai 9104bf7ac6SMatthew Sakai block map era length: 9204bf7ac6SMatthew Sakai The speed with which the block map cache writes out 9304bf7ac6SMatthew Sakai modified block map pages. A smaller era length is likely to 9404bf7ac6SMatthew Sakai reduce the amount of time spent rebuilding, at the cost of 9504bf7ac6SMatthew Sakai increased block map writes during normal operation. The 9604bf7ac6SMatthew Sakai maximum and recommended value is 16380; the minimum value 9704bf7ac6SMatthew Sakai is 1. 9804bf7ac6SMatthew Sakai 9904bf7ac6SMatthew SakaiOptional parameters: 10004bf7ac6SMatthew Sakai-------------------- 10104bf7ac6SMatthew SakaiSome or all of these parameters may be specified as <key> <value> pairs. 10204bf7ac6SMatthew Sakai 10304bf7ac6SMatthew SakaiThread related parameters: 10404bf7ac6SMatthew Sakai 10504bf7ac6SMatthew SakaiDifferent categories of work are assigned to separate thread groups, and 10604bf7ac6SMatthew Sakaithe number of threads in each group can be configured separately. 10704bf7ac6SMatthew Sakai 10804bf7ac6SMatthew SakaiIf <hash>, <logical>, and <physical> are all set to 0, the work handled by 10904bf7ac6SMatthew Sakaiall three thread types will be handled by a single thread. If any of these 11004bf7ac6SMatthew Sakaivalues are non-zero, all of them must be non-zero. 11104bf7ac6SMatthew Sakai 11204bf7ac6SMatthew Sakai ack: 11304bf7ac6SMatthew Sakai The number of threads used to complete bios. Since 11404bf7ac6SMatthew Sakai completing a bio calls an arbitrary completion function 11504bf7ac6SMatthew Sakai outside the vdo volume, threads of this type allow the vdo 11604bf7ac6SMatthew Sakai volume to continue processing requests even when bio 11704bf7ac6SMatthew Sakai completion is slow. The default is 1. 11804bf7ac6SMatthew Sakai 11904bf7ac6SMatthew Sakai bio: 12004bf7ac6SMatthew Sakai The number of threads used to issue bios to the underlying 12104bf7ac6SMatthew Sakai storage. Threads of this type allow the vdo volume to 12204bf7ac6SMatthew Sakai continue processing requests even when bio submission is 12304bf7ac6SMatthew Sakai slow. The default is 4. 12404bf7ac6SMatthew Sakai 12504bf7ac6SMatthew Sakai bioRotationInterval: 12604bf7ac6SMatthew Sakai The number of bios to enqueue on each bio thread before 12704bf7ac6SMatthew Sakai switching to the next thread. The value must be greater 12804bf7ac6SMatthew Sakai than 0 and not more than 1024; the default is 64. 12904bf7ac6SMatthew Sakai 13004bf7ac6SMatthew Sakai cpu: 13104bf7ac6SMatthew Sakai The number of threads used to do CPU-intensive work, such 13204bf7ac6SMatthew Sakai as hashing and compression. The default is 1. 13304bf7ac6SMatthew Sakai 13404bf7ac6SMatthew Sakai hash: 13504bf7ac6SMatthew Sakai The number of threads used to manage data comparisons for 13604bf7ac6SMatthew Sakai deduplication based on the hash value of data blocks. The 13704bf7ac6SMatthew Sakai default is 0. 13804bf7ac6SMatthew Sakai 13904bf7ac6SMatthew Sakai logical: 14004bf7ac6SMatthew Sakai The number of threads used to manage caching and locking 14104bf7ac6SMatthew Sakai based on the logical address of incoming bios. The default 14204bf7ac6SMatthew Sakai is 0; the maximum is 60. 14304bf7ac6SMatthew Sakai 14404bf7ac6SMatthew Sakai physical: 14504bf7ac6SMatthew Sakai The number of threads used to manage administration of the 14604bf7ac6SMatthew Sakai underlying storage device. At format time, a slab size for 14704bf7ac6SMatthew Sakai the vdo is chosen; the vdo storage device must be large 14804bf7ac6SMatthew Sakai enough to have at least 1 slab per physical thread. The 14904bf7ac6SMatthew Sakai default is 0; the maximum is 16. 15004bf7ac6SMatthew Sakai 15104bf7ac6SMatthew SakaiMiscellaneous parameters: 15204bf7ac6SMatthew Sakai 15304bf7ac6SMatthew Sakai maxDiscard: 15404bf7ac6SMatthew Sakai The maximum size of discard bio accepted, in 4096-byte 15504bf7ac6SMatthew Sakai blocks. I/O requests to a vdo volume are normally split 15604bf7ac6SMatthew Sakai into 4096-byte blocks, and processed up to 2048 at a time. 15704bf7ac6SMatthew Sakai However, discard requests to a vdo volume can be 15804bf7ac6SMatthew Sakai automatically split to a larger size, up to <maxDiscard> 15904bf7ac6SMatthew Sakai 4096-byte blocks in a single bio, and are limited to 1500 16004bf7ac6SMatthew Sakai at a time. Increasing this value may provide better overall 16104bf7ac6SMatthew Sakai performance, at the cost of increased latency for the 16204bf7ac6SMatthew Sakai individual discard requests. The default and minimum is 1; 16304bf7ac6SMatthew Sakai the maximum is UINT_MAX / 4096. 16404bf7ac6SMatthew Sakai 16504bf7ac6SMatthew Sakai deduplication: 16604bf7ac6SMatthew Sakai Whether deduplication is enabled. The default is 'on'; the 16704bf7ac6SMatthew Sakai acceptable values are 'on' and 'off'. 16804bf7ac6SMatthew Sakai 16904bf7ac6SMatthew Sakai compression: 17004bf7ac6SMatthew Sakai Whether compression is enabled. The default is 'off'; the 17104bf7ac6SMatthew Sakai acceptable values are 'on' and 'off'. 17204bf7ac6SMatthew Sakai 17304bf7ac6SMatthew SakaiDevice modification 17404bf7ac6SMatthew Sakai------------------- 17504bf7ac6SMatthew Sakai 17604bf7ac6SMatthew SakaiA modified table may be loaded into a running, non-suspended vdo volume. 17704bf7ac6SMatthew SakaiThe modifications will take effect when the device is next resumed. The 17804bf7ac6SMatthew Sakaimodifiable parameters are <logical device size>, <physical device size>, 17904bf7ac6SMatthew Sakai<maxDiscard>, <compression>, and <deduplication>. 18004bf7ac6SMatthew Sakai 18104bf7ac6SMatthew SakaiIf the logical device size or physical device size are changed, upon 18204bf7ac6SMatthew Sakaisuccessful resume vdo will store the new values and require them on future 18304bf7ac6SMatthew Sakaistartups. These two parameters may not be decreased. The logical device 18404bf7ac6SMatthew Sakaisize may not exceed 4 PB. The physical device size must increase by at 18504bf7ac6SMatthew Sakaileast 32832 4096-byte blocks if at all, and must not exceed the size of the 18604bf7ac6SMatthew Sakaiunderlying storage device. Additionally, when formatting the vdo device, a 18704bf7ac6SMatthew Sakaislab size is chosen: the physical device size may never increase above the 18804bf7ac6SMatthew Sakaisize which provides 8192 slabs, and each increase must be large enough to 18904bf7ac6SMatthew Sakaiadd at least one new slab. 19004bf7ac6SMatthew Sakai 19104bf7ac6SMatthew SakaiExamples: 19204bf7ac6SMatthew Sakai 19304bf7ac6SMatthew SakaiStart a previously-formatted vdo volume with 1 GB logical space and 1 GB 19404bf7ac6SMatthew Sakaiphysical space, storing to /dev/dm-1 which has more than 1 GB of space. 19504bf7ac6SMatthew Sakai 19604bf7ac6SMatthew Sakai:: 19704bf7ac6SMatthew Sakai 19804bf7ac6SMatthew Sakai dmsetup create vdo0 --table \ 19904bf7ac6SMatthew Sakai "0 2097152 vdo V4 /dev/dm-1 262144 4096 32768 16380" 20004bf7ac6SMatthew Sakai 20104bf7ac6SMatthew SakaiGrow the logical size to 4 GB. 20204bf7ac6SMatthew Sakai 20304bf7ac6SMatthew Sakai:: 20404bf7ac6SMatthew Sakai 20504bf7ac6SMatthew Sakai dmsetup reload vdo0 --table \ 20604bf7ac6SMatthew Sakai "0 8388608 vdo V4 /dev/dm-1 262144 4096 32768 16380" 20704bf7ac6SMatthew Sakai dmsetup resume vdo0 20804bf7ac6SMatthew Sakai 20904bf7ac6SMatthew SakaiGrow the physical size to 2 GB. 21004bf7ac6SMatthew Sakai 21104bf7ac6SMatthew Sakai:: 21204bf7ac6SMatthew Sakai 21304bf7ac6SMatthew Sakai dmsetup reload vdo0 --table \ 21404bf7ac6SMatthew Sakai "0 8388608 vdo V4 /dev/dm-1 524288 4096 32768 16380" 21504bf7ac6SMatthew Sakai dmsetup resume vdo0 21604bf7ac6SMatthew Sakai 21704bf7ac6SMatthew SakaiGrow the physical size by 1 GB more and increase max discard sectors. 21804bf7ac6SMatthew Sakai 21904bf7ac6SMatthew Sakai:: 22004bf7ac6SMatthew Sakai 22104bf7ac6SMatthew Sakai dmsetup reload vdo0 --table \ 22204bf7ac6SMatthew Sakai "0 10485760 vdo V4 /dev/dm-1 786432 4096 32768 16380 maxDiscard 8" 22304bf7ac6SMatthew Sakai dmsetup resume vdo0 22404bf7ac6SMatthew Sakai 22504bf7ac6SMatthew SakaiStop the vdo volume. 22604bf7ac6SMatthew Sakai 22704bf7ac6SMatthew Sakai:: 22804bf7ac6SMatthew Sakai 22904bf7ac6SMatthew Sakai dmsetup remove vdo0 23004bf7ac6SMatthew Sakai 23104bf7ac6SMatthew SakaiStart the vdo volume again. Note that the logical and physical device sizes 23204bf7ac6SMatthew Sakaimust still match, but other parameters can change. 23304bf7ac6SMatthew Sakai 23404bf7ac6SMatthew Sakai:: 23504bf7ac6SMatthew Sakai 23604bf7ac6SMatthew Sakai dmsetup create vdo1 --table \ 23704bf7ac6SMatthew Sakai "0 10485760 vdo V4 /dev/dm-1 786432 512 65550 5000 hash 1 logical 3 physical 2" 23804bf7ac6SMatthew Sakai 23904bf7ac6SMatthew SakaiMessages 24004bf7ac6SMatthew Sakai-------- 24104bf7ac6SMatthew SakaiAll vdo devices accept messages in the form: 24204bf7ac6SMatthew Sakai 24304bf7ac6SMatthew Sakai:: 24404bf7ac6SMatthew Sakai dmsetup message <target-name> 0 <message-name> <message-parameters> 24504bf7ac6SMatthew Sakai 24604bf7ac6SMatthew SakaiThe messages are: 24704bf7ac6SMatthew Sakai 24804bf7ac6SMatthew Sakai stats: 24904bf7ac6SMatthew Sakai Outputs the current view of the vdo statistics. Mostly used 25004bf7ac6SMatthew Sakai by the vdostats userspace program to interpret the output 25104bf7ac6SMatthew Sakai buffer. 25204bf7ac6SMatthew Sakai 25304bf7ac6SMatthew Sakai dump: 25404bf7ac6SMatthew Sakai Dumps many internal structures to the system log. This is 25504bf7ac6SMatthew Sakai not always safe to run, so it should only be used to debug 25604bf7ac6SMatthew Sakai a hung vdo. Optional parameters to specify structures to 25704bf7ac6SMatthew Sakai dump are: 25804bf7ac6SMatthew Sakai 25904bf7ac6SMatthew Sakai viopool: The pool of I/O requests incoming bios 26004bf7ac6SMatthew Sakai pools: A synonym of 'viopool' 26104bf7ac6SMatthew Sakai vdo: Most of the structures managing on-disk data 26204bf7ac6SMatthew Sakai queues: Basic information about each vdo thread 26304bf7ac6SMatthew Sakai threads: A synonym of 'queues' 26404bf7ac6SMatthew Sakai default: Equivalent to 'queues vdo' 26504bf7ac6SMatthew Sakai all: All of the above. 26604bf7ac6SMatthew Sakai 26704bf7ac6SMatthew Sakai dump-on-shutdown: 26804bf7ac6SMatthew Sakai Perform a default dump next time vdo shuts down. 26904bf7ac6SMatthew Sakai 27004bf7ac6SMatthew Sakai 27104bf7ac6SMatthew SakaiStatus 27204bf7ac6SMatthew Sakai------ 27304bf7ac6SMatthew Sakai 27404bf7ac6SMatthew Sakai:: 27504bf7ac6SMatthew Sakai 27604bf7ac6SMatthew Sakai <device> <operating mode> <in recovery> <index state> 27704bf7ac6SMatthew Sakai <compression state> <physical blocks used> <total physical blocks> 27804bf7ac6SMatthew Sakai 27904bf7ac6SMatthew Sakai device: 28004bf7ac6SMatthew Sakai The name of the vdo volume. 28104bf7ac6SMatthew Sakai 28204bf7ac6SMatthew Sakai operating mode: 28304bf7ac6SMatthew Sakai The current operating mode of the vdo volume; values may be 28404bf7ac6SMatthew Sakai 'normal', 'recovering' (the volume has detected an issue 28504bf7ac6SMatthew Sakai with its metadata and is attempting to repair itself), and 28604bf7ac6SMatthew Sakai 'read-only' (an error has occurred that forces the vdo 28704bf7ac6SMatthew Sakai volume to only support read operations and not writes). 28804bf7ac6SMatthew Sakai 28904bf7ac6SMatthew Sakai in recovery: 29004bf7ac6SMatthew Sakai Whether the vdo volume is currently in recovery mode; 29104bf7ac6SMatthew Sakai values may be 'recovering' or '-' which indicates not 29204bf7ac6SMatthew Sakai recovering. 29304bf7ac6SMatthew Sakai 29404bf7ac6SMatthew Sakai index state: 29504bf7ac6SMatthew Sakai The current state of the deduplication index in the vdo 29604bf7ac6SMatthew Sakai volume; values may be 'closed', 'closing', 'error', 29704bf7ac6SMatthew Sakai 'offline', 'online', 'opening', and 'unknown'. 29804bf7ac6SMatthew Sakai 29904bf7ac6SMatthew Sakai compression state: 30004bf7ac6SMatthew Sakai The current state of compression in the vdo volume; values 30104bf7ac6SMatthew Sakai may be 'offline' and 'online'. 30204bf7ac6SMatthew Sakai 30304bf7ac6SMatthew Sakai used physical blocks: 30404bf7ac6SMatthew Sakai The number of physical blocks in use by the vdo volume. 30504bf7ac6SMatthew Sakai 30604bf7ac6SMatthew Sakai total physical blocks: 30704bf7ac6SMatthew Sakai The total number of physical blocks the vdo volume may use; 30804bf7ac6SMatthew Sakai the difference between this value and the 30904bf7ac6SMatthew Sakai <used physical blocks> is the number of blocks the vdo 31004bf7ac6SMatthew Sakai volume has left before being full. 31104bf7ac6SMatthew Sakai 31204bf7ac6SMatthew SakaiMemory Requirements 31304bf7ac6SMatthew Sakai=================== 31404bf7ac6SMatthew Sakai 31504bf7ac6SMatthew SakaiA vdo target requires a fixed 38 MB of RAM along with the following amounts 31604bf7ac6SMatthew Sakaithat scale with the target: 31704bf7ac6SMatthew Sakai 31804bf7ac6SMatthew Sakai- 1.15 MB of RAM for each 1 MB of configured block map cache size. The 31904bf7ac6SMatthew Sakai block map cache requires a minimum of 150 MB. 32004bf7ac6SMatthew Sakai- 1.6 MB of RAM for each 1 TB of logical space. 32104bf7ac6SMatthew Sakai- 268 MB of RAM for each 1 TB of physical storage managed by the volume. 32204bf7ac6SMatthew Sakai 32304bf7ac6SMatthew SakaiThe deduplication index requires additional memory which scales with the 32404bf7ac6SMatthew Sakaisize of the deduplication window. For dense indexes, the index requires 1 32504bf7ac6SMatthew SakaiGB of RAM per 1 TB of window. For sparse indexes, the index requires 1 GB 32604bf7ac6SMatthew Sakaiof RAM per 10 TB of window. The index configuration is set when the target 32704bf7ac6SMatthew Sakaiis formatted and may not be modified. 32804bf7ac6SMatthew Sakai 329fd5b92b4SKen RaeburnModule Parameters 330fd5b92b4SKen Raeburn================= 331fd5b92b4SKen Raeburn 332fd5b92b4SKen RaeburnThe vdo driver has a numeric parameter 'log_level' which controls the 333fd5b92b4SKen Raeburnverbosity of logging from the driver. The default setting is 6 334fd5b92b4SKen Raeburn(LOGLEVEL_INFO and more severe messages). 335fd5b92b4SKen Raeburn 33604bf7ac6SMatthew SakaiRun-time Usage 33704bf7ac6SMatthew Sakai============== 33804bf7ac6SMatthew Sakai 33904bf7ac6SMatthew SakaiWhen using dm-vdo, it is important to be aware of the ways in which its 34004bf7ac6SMatthew Sakaibehavior differs from other storage targets. 34104bf7ac6SMatthew Sakai 34204bf7ac6SMatthew Sakai- There is no guarantee that over-writes of existing blocks will succeed. 34304bf7ac6SMatthew Sakai Because the underlying storage may be multiply referenced, over-writing 34404bf7ac6SMatthew Sakai an existing block generally requires a vdo to have a free block 34504bf7ac6SMatthew Sakai available. 34604bf7ac6SMatthew Sakai 34704bf7ac6SMatthew Sakai- When blocks are no longer in use, sending a discard request for those 34804bf7ac6SMatthew Sakai blocks lets the vdo release references for those blocks. If the vdo is 34904bf7ac6SMatthew Sakai thinly provisioned, discarding unused blocks is essential to prevent the 35004bf7ac6SMatthew Sakai target from running out of space. However, due to the sharing of 35104bf7ac6SMatthew Sakai duplicate blocks, no discard request for any given logical block is 35204bf7ac6SMatthew Sakai guaranteed to reclaim space. 35304bf7ac6SMatthew Sakai 35404bf7ac6SMatthew Sakai- Assuming the underlying storage properly implements flush requests, vdo 35504bf7ac6SMatthew Sakai is resilient against crashes, however, unflushed writes may or may not 35604bf7ac6SMatthew Sakai persist after a crash. 35704bf7ac6SMatthew Sakai 35804bf7ac6SMatthew Sakai- Each write to a vdo target entails a significant amount of processing. 35904bf7ac6SMatthew Sakai However, much of the work is paralellizable. Therefore, vdo targets 36004bf7ac6SMatthew Sakai achieve better throughput at higher I/O depths, and can support up 2048 36104bf7ac6SMatthew Sakai requests in parallel. 36204bf7ac6SMatthew Sakai 36304bf7ac6SMatthew SakaiTuning 36404bf7ac6SMatthew Sakai====== 36504bf7ac6SMatthew Sakai 36604bf7ac6SMatthew SakaiThe vdo device has many options, and it can be difficult to make optimal 36704bf7ac6SMatthew Sakaichoices without perfect knowledge of the workload. Additionally, most 36804bf7ac6SMatthew Sakaiconfiguration options must be set when a vdo target is started, and cannot 36904bf7ac6SMatthew Sakaibe changed without shutting it down completely; the configuration cannot be 37004bf7ac6SMatthew Sakaichanged while the target is active. Ideally, tuning with simulated 37104bf7ac6SMatthew Sakaiworkloads should be performed before deploying vdo in production 37204bf7ac6SMatthew Sakaienvironments. 37304bf7ac6SMatthew Sakai 37404bf7ac6SMatthew SakaiThe most important value to adjust is the block map cache size. In order to 37504bf7ac6SMatthew Sakaiservice a request for any logical address, a vdo must load the portion of 37604bf7ac6SMatthew Sakaithe block map which holds the relevant mapping. These mappings are cached. 37704bf7ac6SMatthew SakaiPerformance will suffer when the working set does not fit in the cache. By 37804bf7ac6SMatthew Sakaidefault, a vdo allocates 128 MB of metadata cache in RAM to support 37904bf7ac6SMatthew Sakaiefficient access to 100 GB of logical space at a time. It should be scaled 38004bf7ac6SMatthew Sakaiup proportionally for larger working sets. 38104bf7ac6SMatthew Sakai 38204bf7ac6SMatthew SakaiThe logical and physical thread counts should also be adjusted. A logical 38304bf7ac6SMatthew Sakaithread controls a disjoint section of the block map, so additional logical 38404bf7ac6SMatthew Sakaithreads increase parallelism and can increase throughput. Physical threads 38504bf7ac6SMatthew Sakaicontrol a disjoint section of the data blocks, so additional physical 38604bf7ac6SMatthew Sakaithreads can also increase throughput. However, excess threads can waste 38704bf7ac6SMatthew Sakairesources and increase contention. 38804bf7ac6SMatthew Sakai 38904bf7ac6SMatthew SakaiBio submission threads control the parallelism involved in sending I/O to 39004bf7ac6SMatthew Sakaithe underlying storage; fewer threads mean there is more opportunity to 39104bf7ac6SMatthew Sakaireorder I/O requests for performance benefit, but also that each I/O 39204bf7ac6SMatthew Sakairequest has to wait longer before being submitted. 39304bf7ac6SMatthew Sakai 39404bf7ac6SMatthew SakaiBio acknowledgment threads are used for finishing I/O requests. This is 39504bf7ac6SMatthew Sakaidone on dedicated threads since the amount of work required to execute a 39604bf7ac6SMatthew Sakaibio's callback can not be controlled by the vdo itself. Usually one thread 39704bf7ac6SMatthew Sakaiis sufficient but additional threads may be beneficial, particularly when 39804bf7ac6SMatthew Sakaibios have CPU-heavy callbacks. 39904bf7ac6SMatthew Sakai 40004bf7ac6SMatthew SakaiCPU threads are used for hashing and for compression; in workloads with 40104bf7ac6SMatthew Sakaicompression enabled, more threads may result in higher throughput. 40204bf7ac6SMatthew Sakai 40304bf7ac6SMatthew SakaiHash threads are used to sort active requests by hash and determine whether 40404bf7ac6SMatthew Sakaithey should deduplicate; the most CPU intensive actions done by these 40504bf7ac6SMatthew Sakaithreads are comparison of 4096-byte data blocks. In most cases, a single 40604bf7ac6SMatthew Sakaihash thread is sufficient. 407