1ae4a0502SMauro Carvalho Chehab=============================== 2ae4a0502SMauro Carvalho ChehabLIBNVDIMM: Non-Volatile Devices 3ae4a0502SMauro Carvalho Chehab=============================== 4ae4a0502SMauro Carvalho Chehab 5ae4a0502SMauro Carvalho Chehablibnvdimm - kernel / libndctl - userspace helper library 6ae4a0502SMauro Carvalho Chehab 73dd4fe4bSDan Williamsnvdimm@lists.linux.dev 8ae4a0502SMauro Carvalho Chehab 9ae4a0502SMauro Carvalho ChehabVersion 13 10ae4a0502SMauro Carvalho Chehab 11ae4a0502SMauro Carvalho Chehab.. contents: 12ae4a0502SMauro Carvalho Chehab 13ae4a0502SMauro Carvalho Chehab Glossary 14ae4a0502SMauro Carvalho Chehab Overview 15ae4a0502SMauro Carvalho Chehab Supporting Documents 16ae4a0502SMauro Carvalho Chehab Git Trees 17f8669f1dSDan Williams LIBNVDIMM PMEM 18f8669f1dSDan Williams PMEM-REGIONs, Atomic Sectors, and DAX 19ae4a0502SMauro Carvalho Chehab Example NVDIMM Platform 20ae4a0502SMauro Carvalho Chehab LIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API 21ae4a0502SMauro Carvalho Chehab LIBNDCTL: Context 22ae4a0502SMauro Carvalho Chehab libndctl: instantiate a new library context example 23ae4a0502SMauro Carvalho Chehab LIBNVDIMM/LIBNDCTL: Bus 24ae4a0502SMauro Carvalho Chehab libnvdimm: control class device in /sys/class 25ae4a0502SMauro Carvalho Chehab libnvdimm: bus 26ae4a0502SMauro Carvalho Chehab libndctl: bus enumeration example 27ae4a0502SMauro Carvalho Chehab LIBNVDIMM/LIBNDCTL: DIMM (NMEM) 28ae4a0502SMauro Carvalho Chehab libnvdimm: DIMM (NMEM) 29ae4a0502SMauro Carvalho Chehab libndctl: DIMM enumeration example 30ae4a0502SMauro Carvalho Chehab LIBNVDIMM/LIBNDCTL: Region 31ae4a0502SMauro Carvalho Chehab libnvdimm: region 32ae4a0502SMauro Carvalho Chehab libndctl: region enumeration example 33ae4a0502SMauro Carvalho Chehab Why Not Encode the Region Type into the Region Name? 34ae4a0502SMauro Carvalho Chehab How Do I Determine the Major Type of a Region? 35ae4a0502SMauro Carvalho Chehab LIBNVDIMM/LIBNDCTL: Namespace 36ae4a0502SMauro Carvalho Chehab libnvdimm: namespace 37ae4a0502SMauro Carvalho Chehab libndctl: namespace enumeration example 38ae4a0502SMauro Carvalho Chehab libndctl: namespace creation example 39ae4a0502SMauro Carvalho Chehab Why the Term "namespace"? 40ae4a0502SMauro Carvalho Chehab LIBNVDIMM/LIBNDCTL: Block Translation Table "btt" 41ae4a0502SMauro Carvalho Chehab libnvdimm: btt layout 42ae4a0502SMauro Carvalho Chehab libndctl: btt creation example 43ae4a0502SMauro Carvalho Chehab Summary LIBNDCTL Diagram 44ae4a0502SMauro Carvalho Chehab 45ae4a0502SMauro Carvalho Chehab 46ae4a0502SMauro Carvalho ChehabGlossary 47ae4a0502SMauro Carvalho Chehab======== 48ae4a0502SMauro Carvalho Chehab 49ae4a0502SMauro Carvalho ChehabPMEM: 50ae4a0502SMauro Carvalho Chehab A system-physical-address range where writes are persistent. A 51ae4a0502SMauro Carvalho Chehab block device composed of PMEM is capable of DAX. A PMEM address range 52ae4a0502SMauro Carvalho Chehab may span an interleave of several DIMMs. 53ae4a0502SMauro Carvalho Chehab 54ae4a0502SMauro Carvalho ChehabDPA: 55ae4a0502SMauro Carvalho Chehab DIMM Physical Address, is a DIMM-relative offset. With one DIMM in 56ae4a0502SMauro Carvalho Chehab the system there would be a 1:1 system-physical-address:DPA association. 57ae4a0502SMauro Carvalho Chehab Once more DIMMs are added a memory controller interleave must be 58ae4a0502SMauro Carvalho Chehab decoded to determine the DPA associated with a given 59f8669f1dSDan Williams system-physical-address. 60ae4a0502SMauro Carvalho Chehab 61ae4a0502SMauro Carvalho ChehabDAX: 62ae4a0502SMauro Carvalho Chehab File system extensions to bypass the page cache and block layer to 63ae4a0502SMauro Carvalho Chehab mmap persistent memory, from a PMEM block device, directly into a 64ae4a0502SMauro Carvalho Chehab process address space. 65ae4a0502SMauro Carvalho Chehab 66ae4a0502SMauro Carvalho ChehabDSM: 676db35a24SRandy Dunlap Device Specific Method: ACPI method to control specific 68ae4a0502SMauro Carvalho Chehab device - in this case the firmware. 69ae4a0502SMauro Carvalho Chehab 70ae4a0502SMauro Carvalho ChehabDCR: 71ae4a0502SMauro Carvalho Chehab NVDIMM Control Region Structure defined in ACPI 6 Section 5.2.25.5. 72ae4a0502SMauro Carvalho Chehab It defines a vendor-id, device-id, and interface format for a given DIMM. 73ae4a0502SMauro Carvalho Chehab 74ae4a0502SMauro Carvalho ChehabBTT: 75ae4a0502SMauro Carvalho Chehab Block Translation Table: Persistent memory is byte addressable. 76ae4a0502SMauro Carvalho Chehab Existing software may have an expectation that the power-fail-atomicity 77ae4a0502SMauro Carvalho Chehab of writes is at least one sector, 512 bytes. The BTT is an indirection 78f8669f1dSDan Williams table with atomic update semantics to front a PMEM block device 79ae4a0502SMauro Carvalho Chehab driver and present arbitrary atomic sector sizes. 80ae4a0502SMauro Carvalho Chehab 81ae4a0502SMauro Carvalho ChehabLABEL: 82ae4a0502SMauro Carvalho Chehab Metadata stored on a DIMM device that partitions and identifies 83f8669f1dSDan Williams (persistently names) capacity allocated to different PMEM namespaces. It 84f8669f1dSDan Williams also indicates whether an address abstraction like a BTT is applied to 85*7852fe3aSRandy Dunlap the namespace. Note that traditional partition tables, GPT/MBR, are 86f8669f1dSDan Williams layered on top of a PMEM namespace, or an address abstraction like BTT 87f8669f1dSDan Williams if present, but partition support is deprecated going forward. 88ae4a0502SMauro Carvalho Chehab 89ae4a0502SMauro Carvalho Chehab 90ae4a0502SMauro Carvalho ChehabOverview 91ae4a0502SMauro Carvalho Chehab======== 92ae4a0502SMauro Carvalho Chehab 93f8669f1dSDan WilliamsThe LIBNVDIMM subsystem provides support for PMEM described by platform 94f8669f1dSDan Williamsfirmware or a device driver. On ACPI based systems the platform firmware 95f8669f1dSDan Williamsconveys persistent memory resource via the ACPI NFIT "NVDIMM Firmware 96f8669f1dSDan WilliamsInterface Table" in ACPI 6. While the LIBNVDIMM subsystem implementation 97f8669f1dSDan Williamsis generic and supports pre-NFIT platforms, it was guided by the 98f8669f1dSDan Williamssuperset of capabilities need to support this ACPI 6 definition for 99f8669f1dSDan WilliamsNVDIMM resources. The original implementation supported the 100f8669f1dSDan Williamsblock-window-aperture capability described in the NFIT, but that support 101f8669f1dSDan Williamshas since been abandoned and never shipped in a product. 102ae4a0502SMauro Carvalho Chehab 103ae4a0502SMauro Carvalho ChehabSupporting Documents 104ae4a0502SMauro Carvalho Chehab-------------------- 105ae4a0502SMauro Carvalho Chehab 106ae4a0502SMauro Carvalho ChehabACPI 6: 107d3603f4cSAlexander A. Klimov https://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf 108ae4a0502SMauro Carvalho ChehabNVDIMM Namespace: 109d3603f4cSAlexander A. Klimov https://pmem.io/documents/NVDIMM_Namespace_Spec.pdf 110ae4a0502SMauro Carvalho ChehabDSM Interface Example: 111d3603f4cSAlexander A. Klimov https://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf 112ae4a0502SMauro Carvalho ChehabDriver Writer's Guide: 113d3603f4cSAlexander A. Klimov https://pmem.io/documents/NVDIMM_Driver_Writers_Guide.pdf 114ae4a0502SMauro Carvalho Chehab 115ae4a0502SMauro Carvalho ChehabGit Trees 116ae4a0502SMauro Carvalho Chehab--------- 117ae4a0502SMauro Carvalho Chehab 118ae4a0502SMauro Carvalho ChehabLIBNVDIMM: 119f8669f1dSDan Williams https://git.kernel.org/cgit/linux/kernel/git/nvdimm/nvdimm.git 120ae4a0502SMauro Carvalho ChehabLIBNDCTL: 121ae4a0502SMauro Carvalho Chehab https://github.com/pmem/ndctl.git 122ae4a0502SMauro Carvalho Chehab 123ae4a0502SMauro Carvalho Chehab 124f8669f1dSDan WilliamsLIBNVDIMM PMEM 125f8669f1dSDan Williams============== 126ae4a0502SMauro Carvalho Chehab 127ae4a0502SMauro Carvalho ChehabPrior to the arrival of the NFIT, non-volatile memory was described to a 128ae4a0502SMauro Carvalho Chehabsystem in various ad-hoc ways. Usually only the bare minimum was 129ae4a0502SMauro Carvalho Chehabprovided, namely, a single system-physical-address range where writes 130ae4a0502SMauro Carvalho Chehabare expected to be durable after a system power loss. Now, the NFIT 131ae4a0502SMauro Carvalho Chehabspecification standardizes not only the description of PMEM, but also 132f8669f1dSDan Williamsplatform message-passing entry points for control and configuration. 133ae4a0502SMauro Carvalho Chehab 134f8669f1dSDan WilliamsPMEM (nd_pmem.ko): Drives a system-physical-address range. This range is 135f8669f1dSDan Williamscontiguous in system memory and may be interleaved (hardware memory controller 136f8669f1dSDan Williamsstriped) across multiple DIMMs. When interleaved the platform may optionally 137f8669f1dSDan Williamsprovide details of which DIMMs are participating in the interleave. 138ae4a0502SMauro Carvalho Chehab 139f8669f1dSDan WilliamsIt is worth noting that when the labeling capability is detected (a EFI 140f8669f1dSDan Williamsnamespace label index block is found), then no block device is created 141f8669f1dSDan Williamsby default as userspace needs to do at least one allocation of DPA to 142f8669f1dSDan Williamsthe PMEM range. In contrast ND_NAMESPACE_IO ranges, once registered, 143f8669f1dSDan Williamscan be immediately attached to nd_pmem. This latter mode is called 144f8669f1dSDan Williamslabel-less or "legacy". 145ae4a0502SMauro Carvalho Chehab 146f8669f1dSDan WilliamsPMEM-REGIONs, Atomic Sectors, and DAX 147ada8d8d3STom Rix------------------------------------- 148ae4a0502SMauro Carvalho Chehab 149f8669f1dSDan WilliamsFor the cases where an application or filesystem still needs atomic sector 150f8669f1dSDan Williamsupdate guarantees it can register a BTT on a PMEM device or partition. See 151ae4a0502SMauro Carvalho ChehabLIBNVDIMM/NDCTL: Block Translation Table "btt" 152ae4a0502SMauro Carvalho Chehab 153ae4a0502SMauro Carvalho Chehab 154ae4a0502SMauro Carvalho ChehabExample NVDIMM Platform 155ae4a0502SMauro Carvalho Chehab======================= 156ae4a0502SMauro Carvalho Chehab 157ae4a0502SMauro Carvalho ChehabFor the remainder of this document the following diagram will be 158ae4a0502SMauro Carvalho Chehabreferenced for any example sysfs layouts:: 159ae4a0502SMauro Carvalho Chehab 160ae4a0502SMauro Carvalho Chehab 161f8669f1dSDan Williams (a) (b) DIMM 162ae4a0502SMauro Carvalho Chehab +-------------------+--------+--------+--------+ 163f8669f1dSDan Williams +------+ | pm0.0 | free | pm1.0 | free | 0 164ae4a0502SMauro Carvalho Chehab | imc0 +--+- - - region0- - - +--------+ +--------+ 165f8669f1dSDan Williams +--+---+ | pm0.0 | free | pm1.0 | free | 1 166ae4a0502SMauro Carvalho Chehab | +-------------------+--------v v--------+ 167ae4a0502SMauro Carvalho Chehab +--+---+ | | 168ae4a0502SMauro Carvalho Chehab | cpu0 | region1 169ae4a0502SMauro Carvalho Chehab +--+---+ | | 170ae4a0502SMauro Carvalho Chehab | +----------------------------^ ^--------+ 171f8669f1dSDan Williams +--+---+ | free | pm1.0 | free | 2 172ae4a0502SMauro Carvalho Chehab | imc1 +--+----------------------------| +--------+ 173f8669f1dSDan Williams +------+ | free | pm1.0 | free | 3 174ae4a0502SMauro Carvalho Chehab +----------------------------+--------+--------+ 175ae4a0502SMauro Carvalho Chehab 176ae4a0502SMauro Carvalho ChehabIn this platform we have four DIMMs and two memory controllers in one 177f8669f1dSDan Williamssocket. Each PMEM interleave set is identified by a region device with 178f8669f1dSDan Williamsa dynamically assigned id. 179ae4a0502SMauro Carvalho Chehab 180ae4a0502SMauro Carvalho Chehab 1. The first portion of DIMM0 and DIMM1 are interleaved as REGION0. A 181ae4a0502SMauro Carvalho Chehab single PMEM namespace is created in the REGION0-SPA-range that spans most 182ae4a0502SMauro Carvalho Chehab of DIMM0 and DIMM1 with a user-specified name of "pm0.0". Some of that 183f8669f1dSDan Williams interleaved system-physical-address range is left free for 184f8669f1dSDan Williams another PMEM namespace to be defined. 185ae4a0502SMauro Carvalho Chehab 186ae4a0502SMauro Carvalho Chehab 2. In the last portion of DIMM0 and DIMM1 we have an interleaved 187ae4a0502SMauro Carvalho Chehab system-physical-address range, REGION1, that spans those two DIMMs as 188ae4a0502SMauro Carvalho Chehab well as DIMM2 and DIMM3. Some of REGION1 is allocated to a PMEM namespace 189f8669f1dSDan Williams named "pm1.0". 190ae4a0502SMauro Carvalho Chehab 191ae4a0502SMauro Carvalho Chehab This bus is provided by the kernel under the device 1921f5ea872SMichal Suchanek /sys/devices/platform/nfit_test.0 when the nfit_test.ko module from 193f8669f1dSDan Williams tools/testing/nvdimm is loaded. This module is a unit test for 194f8669f1dSDan Williams LIBNVDIMM and the acpi_nfit.ko driver. 195ae4a0502SMauro Carvalho Chehab 196ae4a0502SMauro Carvalho Chehab 197ae4a0502SMauro Carvalho ChehabLIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API 198ae4a0502SMauro Carvalho Chehab======================================================== 199ae4a0502SMauro Carvalho Chehab 200ae4a0502SMauro Carvalho ChehabWhat follows is a description of the LIBNVDIMM sysfs layout and a 201ae4a0502SMauro Carvalho Chehabcorresponding object hierarchy diagram as viewed through the LIBNDCTL 202ae4a0502SMauro Carvalho ChehabAPI. The example sysfs paths and diagrams are relative to the Example 203ae4a0502SMauro Carvalho ChehabNVDIMM Platform which is also the LIBNVDIMM bus used in the LIBNDCTL unit 204ae4a0502SMauro Carvalho Chehabtest. 205ae4a0502SMauro Carvalho Chehab 206ae4a0502SMauro Carvalho ChehabLIBNDCTL: Context 207ae4a0502SMauro Carvalho Chehab----------------- 208ae4a0502SMauro Carvalho Chehab 209ae4a0502SMauro Carvalho ChehabEvery API call in the LIBNDCTL library requires a context that holds the 210ae4a0502SMauro Carvalho Chehablogging parameters and other library instance state. The library is 211ae4a0502SMauro Carvalho Chehabbased on the libabc template: 212ae4a0502SMauro Carvalho Chehab 213ae4a0502SMauro Carvalho Chehab https://git.kernel.org/cgit/linux/kernel/git/kay/libabc.git 214ae4a0502SMauro Carvalho Chehab 215ae4a0502SMauro Carvalho ChehabLIBNDCTL: instantiate a new library context example 216ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 217ae4a0502SMauro Carvalho Chehab 218ae4a0502SMauro Carvalho Chehab:: 219ae4a0502SMauro Carvalho Chehab 220ae4a0502SMauro Carvalho Chehab struct ndctl_ctx *ctx; 221ae4a0502SMauro Carvalho Chehab 222ae4a0502SMauro Carvalho Chehab if (ndctl_new(&ctx) == 0) 223ae4a0502SMauro Carvalho Chehab return ctx; 224ae4a0502SMauro Carvalho Chehab else 225ae4a0502SMauro Carvalho Chehab return NULL; 226ae4a0502SMauro Carvalho Chehab 227ae4a0502SMauro Carvalho ChehabLIBNVDIMM/LIBNDCTL: Bus 228ae4a0502SMauro Carvalho Chehab----------------------- 229ae4a0502SMauro Carvalho Chehab 230ae4a0502SMauro Carvalho ChehabA bus has a 1:1 relationship with an NFIT. The current expectation for 231ae4a0502SMauro Carvalho ChehabACPI based systems is that there is only ever one platform-global NFIT. 232ae4a0502SMauro Carvalho ChehabThat said, it is trivial to register multiple NFITs, the specification 233ae4a0502SMauro Carvalho Chehabdoes not preclude it. The infrastructure supports multiple busses and 234ae4a0502SMauro Carvalho Chehabwe use this capability to test multiple NFIT configurations in the unit 235ae4a0502SMauro Carvalho Chehabtest. 236ae4a0502SMauro Carvalho Chehab 237ae4a0502SMauro Carvalho ChehabLIBNVDIMM: control class device in /sys/class 238ae4a0502SMauro Carvalho Chehab--------------------------------------------- 239ae4a0502SMauro Carvalho Chehab 240ae4a0502SMauro Carvalho ChehabThis character device accepts DSM messages to be passed to DIMM 241ae4a0502SMauro Carvalho Chehabidentified by its NFIT handle:: 242ae4a0502SMauro Carvalho Chehab 243ae4a0502SMauro Carvalho Chehab /sys/class/nd/ndctl0 244ae4a0502SMauro Carvalho Chehab |-- dev 245ae4a0502SMauro Carvalho Chehab |-- device -> ../../../ndbus0 246ae4a0502SMauro Carvalho Chehab |-- subsystem -> ../../../../../../../class/nd 247ae4a0502SMauro Carvalho Chehab 248ae4a0502SMauro Carvalho Chehab 249ae4a0502SMauro Carvalho Chehab 250ae4a0502SMauro Carvalho ChehabLIBNVDIMM: bus 251ae4a0502SMauro Carvalho Chehab-------------- 252ae4a0502SMauro Carvalho Chehab 253ae4a0502SMauro Carvalho Chehab:: 254ae4a0502SMauro Carvalho Chehab 255ae4a0502SMauro Carvalho Chehab struct nvdimm_bus *nvdimm_bus_register(struct device *parent, 256ae4a0502SMauro Carvalho Chehab struct nvdimm_bus_descriptor *nfit_desc); 257ae4a0502SMauro Carvalho Chehab 258ae4a0502SMauro Carvalho Chehab:: 259ae4a0502SMauro Carvalho Chehab 260ae4a0502SMauro Carvalho Chehab /sys/devices/platform/nfit_test.0/ndbus0 261ae4a0502SMauro Carvalho Chehab |-- commands 262ae4a0502SMauro Carvalho Chehab |-- nd 263ae4a0502SMauro Carvalho Chehab |-- nfit 264ae4a0502SMauro Carvalho Chehab |-- nmem0 265ae4a0502SMauro Carvalho Chehab |-- nmem1 266ae4a0502SMauro Carvalho Chehab |-- nmem2 267ae4a0502SMauro Carvalho Chehab |-- nmem3 268ae4a0502SMauro Carvalho Chehab |-- power 269ae4a0502SMauro Carvalho Chehab |-- provider 270ae4a0502SMauro Carvalho Chehab |-- region0 271ae4a0502SMauro Carvalho Chehab |-- region1 272ae4a0502SMauro Carvalho Chehab |-- region2 273ae4a0502SMauro Carvalho Chehab |-- region3 274ae4a0502SMauro Carvalho Chehab |-- region4 275ae4a0502SMauro Carvalho Chehab |-- region5 276ae4a0502SMauro Carvalho Chehab |-- uevent 277ae4a0502SMauro Carvalho Chehab `-- wait_probe 278ae4a0502SMauro Carvalho Chehab 279ae4a0502SMauro Carvalho ChehabLIBNDCTL: bus enumeration example 280ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 281ae4a0502SMauro Carvalho Chehab 282ae4a0502SMauro Carvalho ChehabFind the bus handle that describes the bus from Example NVDIMM Platform:: 283ae4a0502SMauro Carvalho Chehab 284ae4a0502SMauro Carvalho Chehab static struct ndctl_bus *get_bus_by_provider(struct ndctl_ctx *ctx, 285ae4a0502SMauro Carvalho Chehab const char *provider) 286ae4a0502SMauro Carvalho Chehab { 287ae4a0502SMauro Carvalho Chehab struct ndctl_bus *bus; 288ae4a0502SMauro Carvalho Chehab 289ae4a0502SMauro Carvalho Chehab ndctl_bus_foreach(ctx, bus) 290ae4a0502SMauro Carvalho Chehab if (strcmp(provider, ndctl_bus_get_provider(bus)) == 0) 291ae4a0502SMauro Carvalho Chehab return bus; 292ae4a0502SMauro Carvalho Chehab 293ae4a0502SMauro Carvalho Chehab return NULL; 294ae4a0502SMauro Carvalho Chehab } 295ae4a0502SMauro Carvalho Chehab 296ae4a0502SMauro Carvalho Chehab bus = get_bus_by_provider(ctx, "nfit_test.0"); 297ae4a0502SMauro Carvalho Chehab 298ae4a0502SMauro Carvalho Chehab 299ae4a0502SMauro Carvalho ChehabLIBNVDIMM/LIBNDCTL: DIMM (NMEM) 300ae4a0502SMauro Carvalho Chehab------------------------------- 301ae4a0502SMauro Carvalho Chehab 302ae4a0502SMauro Carvalho ChehabThe DIMM device provides a character device for sending commands to 303ae4a0502SMauro Carvalho Chehabhardware, and it is a container for LABELs. If the DIMM is defined by 304ae4a0502SMauro Carvalho ChehabNFIT then an optional 'nfit' attribute sub-directory is available to add 305ae4a0502SMauro Carvalho ChehabNFIT-specifics. 306ae4a0502SMauro Carvalho Chehab 307ae4a0502SMauro Carvalho ChehabNote that the kernel device name for "DIMMs" is "nmemX". The NFIT 308ae4a0502SMauro Carvalho Chehabdescribes these devices via "Memory Device to System Physical Address 309ae4a0502SMauro Carvalho ChehabRange Mapping Structure", and there is no requirement that they actually 310ae4a0502SMauro Carvalho Chehabbe physical DIMMs, so we use a more generic name. 311ae4a0502SMauro Carvalho Chehab 312ae4a0502SMauro Carvalho ChehabLIBNVDIMM: DIMM (NMEM) 313ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^ 314ae4a0502SMauro Carvalho Chehab 315ae4a0502SMauro Carvalho Chehab:: 316ae4a0502SMauro Carvalho Chehab 317ae4a0502SMauro Carvalho Chehab struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data, 318ae4a0502SMauro Carvalho Chehab const struct attribute_group **groups, unsigned long flags, 319ae4a0502SMauro Carvalho Chehab unsigned long *dsm_mask); 320ae4a0502SMauro Carvalho Chehab 321ae4a0502SMauro Carvalho Chehab:: 322ae4a0502SMauro Carvalho Chehab 323ae4a0502SMauro Carvalho Chehab /sys/devices/platform/nfit_test.0/ndbus0 324ae4a0502SMauro Carvalho Chehab |-- nmem0 325ae4a0502SMauro Carvalho Chehab | |-- available_slots 326ae4a0502SMauro Carvalho Chehab | |-- commands 327ae4a0502SMauro Carvalho Chehab | |-- dev 328ae4a0502SMauro Carvalho Chehab | |-- devtype 329ae4a0502SMauro Carvalho Chehab | |-- driver -> ../../../../../bus/nd/drivers/nvdimm 330ae4a0502SMauro Carvalho Chehab | |-- modalias 331ae4a0502SMauro Carvalho Chehab | |-- nfit 332ae4a0502SMauro Carvalho Chehab | | |-- device 333ae4a0502SMauro Carvalho Chehab | | |-- format 334ae4a0502SMauro Carvalho Chehab | | |-- handle 335ae4a0502SMauro Carvalho Chehab | | |-- phys_id 336ae4a0502SMauro Carvalho Chehab | | |-- rev_id 337ae4a0502SMauro Carvalho Chehab | | |-- serial 338ae4a0502SMauro Carvalho Chehab | | `-- vendor 339ae4a0502SMauro Carvalho Chehab | |-- state 340ae4a0502SMauro Carvalho Chehab | |-- subsystem -> ../../../../../bus/nd 341ae4a0502SMauro Carvalho Chehab | `-- uevent 342ae4a0502SMauro Carvalho Chehab |-- nmem1 343ae4a0502SMauro Carvalho Chehab [..] 344ae4a0502SMauro Carvalho Chehab 345ae4a0502SMauro Carvalho Chehab 346ae4a0502SMauro Carvalho ChehabLIBNDCTL: DIMM enumeration example 347ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 348ae4a0502SMauro Carvalho Chehab 349ae4a0502SMauro Carvalho ChehabNote, in this example we are assuming NFIT-defined DIMMs which are 350ae4a0502SMauro Carvalho Chehabidentified by an "nfit_handle" a 32-bit value where: 351ae4a0502SMauro Carvalho Chehab 352ae4a0502SMauro Carvalho Chehab - Bit 3:0 DIMM number within the memory channel 353ae4a0502SMauro Carvalho Chehab - Bit 7:4 memory channel number 354ae4a0502SMauro Carvalho Chehab - Bit 11:8 memory controller ID 355ae4a0502SMauro Carvalho Chehab - Bit 15:12 socket ID (within scope of a Node controller if node 356ae4a0502SMauro Carvalho Chehab controller is present) 357ae4a0502SMauro Carvalho Chehab - Bit 27:16 Node Controller ID 358ae4a0502SMauro Carvalho Chehab - Bit 31:28 Reserved 359ae4a0502SMauro Carvalho Chehab 360ae4a0502SMauro Carvalho Chehab:: 361ae4a0502SMauro Carvalho Chehab 362ae4a0502SMauro Carvalho Chehab static struct ndctl_dimm *get_dimm_by_handle(struct ndctl_bus *bus, 363ae4a0502SMauro Carvalho Chehab unsigned int handle) 364ae4a0502SMauro Carvalho Chehab { 365ae4a0502SMauro Carvalho Chehab struct ndctl_dimm *dimm; 366ae4a0502SMauro Carvalho Chehab 367ae4a0502SMauro Carvalho Chehab ndctl_dimm_foreach(bus, dimm) 368ae4a0502SMauro Carvalho Chehab if (ndctl_dimm_get_handle(dimm) == handle) 369ae4a0502SMauro Carvalho Chehab return dimm; 370ae4a0502SMauro Carvalho Chehab 371ae4a0502SMauro Carvalho Chehab return NULL; 372ae4a0502SMauro Carvalho Chehab } 373ae4a0502SMauro Carvalho Chehab 374ae4a0502SMauro Carvalho Chehab #define DIMM_HANDLE(n, s, i, c, d) \ 375ae4a0502SMauro Carvalho Chehab (((n & 0xfff) << 16) | ((s & 0xf) << 12) | ((i & 0xf) << 8) \ 376ae4a0502SMauro Carvalho Chehab | ((c & 0xf) << 4) | (d & 0xf)) 377ae4a0502SMauro Carvalho Chehab 378ae4a0502SMauro Carvalho Chehab dimm = get_dimm_by_handle(bus, DIMM_HANDLE(0, 0, 0, 0, 0)); 379ae4a0502SMauro Carvalho Chehab 380ae4a0502SMauro Carvalho ChehabLIBNVDIMM/LIBNDCTL: Region 381ae4a0502SMauro Carvalho Chehab-------------------------- 382ae4a0502SMauro Carvalho Chehab 383f8669f1dSDan WilliamsA generic REGION device is registered for each PMEM interleave-set / 384f8669f1dSDan Williamsrange. Per the example there are 2 PMEM regions on the "nfit_test.0" 385f8669f1dSDan Williamsbus. The primary role of regions are to be a container of "mappings". A 386f8669f1dSDan Williamsmapping is a tuple of <DIMM, DPA-start-offset, length>. 387ae4a0502SMauro Carvalho Chehab 388f8669f1dSDan WilliamsLIBNVDIMM provides a built-in driver for REGION devices. This driver 389f8669f1dSDan Williamsis responsible for all parsing LABELs, if present, and then emitting NAMESPACE 390f8669f1dSDan Williamsdevices for the nd_pmem driver to consume. 391ae4a0502SMauro Carvalho Chehab 392ae4a0502SMauro Carvalho ChehabIn addition to the generic attributes of "mapping"s, "interleave_ways" 393ae4a0502SMauro Carvalho Chehaband "size" the REGION device also exports some convenience attributes. 394ae4a0502SMauro Carvalho Chehab"nstype" indicates the integer type of namespace-device this region 395ae4a0502SMauro Carvalho Chehabemits, "devtype" duplicates the DEVTYPE variable stored by udev at the 396ae4a0502SMauro Carvalho Chehab'add' event, "modalias" duplicates the MODALIAS variable stored by udev 397ae4a0502SMauro Carvalho Chehabat the 'add' event, and finally, the optional "spa_index" is provided in 398ae4a0502SMauro Carvalho Chehabthe case where the region is defined by a SPA. 399ae4a0502SMauro Carvalho Chehab 400ae4a0502SMauro Carvalho ChehabLIBNVDIMM: region:: 401ae4a0502SMauro Carvalho Chehab 402ae4a0502SMauro Carvalho Chehab struct nd_region *nvdimm_pmem_region_create(struct nvdimm_bus *nvdimm_bus, 403ae4a0502SMauro Carvalho Chehab struct nd_region_desc *ndr_desc); 404ae4a0502SMauro Carvalho Chehab 405ae4a0502SMauro Carvalho Chehab:: 406ae4a0502SMauro Carvalho Chehab 407ae4a0502SMauro Carvalho Chehab /sys/devices/platform/nfit_test.0/ndbus0 408ae4a0502SMauro Carvalho Chehab |-- region0 409ae4a0502SMauro Carvalho Chehab | |-- available_size 410ae4a0502SMauro Carvalho Chehab | |-- btt0 411ae4a0502SMauro Carvalho Chehab | |-- btt_seed 412ae4a0502SMauro Carvalho Chehab | |-- devtype 413ae4a0502SMauro Carvalho Chehab | |-- driver -> ../../../../../bus/nd/drivers/nd_region 414ae4a0502SMauro Carvalho Chehab | |-- init_namespaces 415ae4a0502SMauro Carvalho Chehab | |-- mapping0 416ae4a0502SMauro Carvalho Chehab | |-- mapping1 417ae4a0502SMauro Carvalho Chehab | |-- mappings 418ae4a0502SMauro Carvalho Chehab | |-- modalias 419ae4a0502SMauro Carvalho Chehab | |-- namespace0.0 420ae4a0502SMauro Carvalho Chehab | |-- namespace_seed 421ae4a0502SMauro Carvalho Chehab | |-- numa_node 422ae4a0502SMauro Carvalho Chehab | |-- nfit 423ae4a0502SMauro Carvalho Chehab | | `-- spa_index 424ae4a0502SMauro Carvalho Chehab | |-- nstype 425ae4a0502SMauro Carvalho Chehab | |-- set_cookie 426ae4a0502SMauro Carvalho Chehab | |-- size 427ae4a0502SMauro Carvalho Chehab | |-- subsystem -> ../../../../../bus/nd 428ae4a0502SMauro Carvalho Chehab | `-- uevent 429ae4a0502SMauro Carvalho Chehab |-- region1 430ae4a0502SMauro Carvalho Chehab [..] 431ae4a0502SMauro Carvalho Chehab 432ae4a0502SMauro Carvalho ChehabLIBNDCTL: region enumeration example 433ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 434ae4a0502SMauro Carvalho Chehab 435ae4a0502SMauro Carvalho ChehabSample region retrieval routines based on NFIT-unique data like 436f8669f1dSDan Williams"spa_index" (interleave set id). 437f8669f1dSDan Williams 438f8669f1dSDan Williams:: 439ae4a0502SMauro Carvalho Chehab 440ae4a0502SMauro Carvalho Chehab static struct ndctl_region *get_pmem_region_by_spa_index(struct ndctl_bus *bus, 441ae4a0502SMauro Carvalho Chehab unsigned int spa_index) 442ae4a0502SMauro Carvalho Chehab { 443ae4a0502SMauro Carvalho Chehab struct ndctl_region *region; 444ae4a0502SMauro Carvalho Chehab 445ae4a0502SMauro Carvalho Chehab ndctl_region_foreach(bus, region) { 446ae4a0502SMauro Carvalho Chehab if (ndctl_region_get_type(region) != ND_DEVICE_REGION_PMEM) 447ae4a0502SMauro Carvalho Chehab continue; 448ae4a0502SMauro Carvalho Chehab if (ndctl_region_get_spa_index(region) == spa_index) 449ae4a0502SMauro Carvalho Chehab return region; 450ae4a0502SMauro Carvalho Chehab } 451ae4a0502SMauro Carvalho Chehab return NULL; 452ae4a0502SMauro Carvalho Chehab } 453ae4a0502SMauro Carvalho Chehab 454ae4a0502SMauro Carvalho Chehab 455ae4a0502SMauro Carvalho ChehabLIBNVDIMM/LIBNDCTL: Namespace 456ae4a0502SMauro Carvalho Chehab----------------------------- 457ae4a0502SMauro Carvalho Chehab 458f8669f1dSDan WilliamsA REGION, after resolving DPA aliasing and LABEL specified boundaries, surfaces 459f8669f1dSDan Williamsone or more "namespace" devices. The arrival of a "namespace" device currently 460f8669f1dSDan Williamstriggers the nd_pmem driver to load and register a disk/block device. 461ae4a0502SMauro Carvalho Chehab 462ae4a0502SMauro Carvalho ChehabLIBNVDIMM: namespace 463ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^ 464ae4a0502SMauro Carvalho Chehab 465f8669f1dSDan WilliamsHere is a sample layout from the 2 major types of NAMESPACE where namespace0.0 466f8669f1dSDan Williamsrepresents DIMM-info-backed PMEM (note that it has a 'uuid' attribute), and 467f8669f1dSDan Williamsnamespace1.0 represents an anonymous PMEM namespace (note that has no 'uuid' 468f8669f1dSDan Williamsattribute due to not support a LABEL) 469f8669f1dSDan Williams 470f8669f1dSDan Williams:: 471ae4a0502SMauro Carvalho Chehab 472ae4a0502SMauro Carvalho Chehab /sys/devices/platform/nfit_test.0/ndbus0/region0/namespace0.0 473ae4a0502SMauro Carvalho Chehab |-- alt_name 474ae4a0502SMauro Carvalho Chehab |-- devtype 475ae4a0502SMauro Carvalho Chehab |-- dpa_extents 476ae4a0502SMauro Carvalho Chehab |-- force_raw 477ae4a0502SMauro Carvalho Chehab |-- modalias 478ae4a0502SMauro Carvalho Chehab |-- numa_node 479ae4a0502SMauro Carvalho Chehab |-- resource 480ae4a0502SMauro Carvalho Chehab |-- size 481ae4a0502SMauro Carvalho Chehab |-- subsystem -> ../../../../../../bus/nd 482ae4a0502SMauro Carvalho Chehab |-- type 483ae4a0502SMauro Carvalho Chehab |-- uevent 484ae4a0502SMauro Carvalho Chehab `-- uuid 485f8669f1dSDan Williams /sys/devices/platform/nfit_test.1/ndbus1/region1/namespace1.0 486ae4a0502SMauro Carvalho Chehab |-- block 487ae4a0502SMauro Carvalho Chehab | `-- pmem0 488ae4a0502SMauro Carvalho Chehab |-- devtype 489ae4a0502SMauro Carvalho Chehab |-- driver -> ../../../../../../bus/nd/drivers/pmem 490ae4a0502SMauro Carvalho Chehab |-- force_raw 491ae4a0502SMauro Carvalho Chehab |-- modalias 492ae4a0502SMauro Carvalho Chehab |-- numa_node 493ae4a0502SMauro Carvalho Chehab |-- resource 494ae4a0502SMauro Carvalho Chehab |-- size 495ae4a0502SMauro Carvalho Chehab |-- subsystem -> ../../../../../../bus/nd 496ae4a0502SMauro Carvalho Chehab |-- type 497ae4a0502SMauro Carvalho Chehab `-- uevent 498ae4a0502SMauro Carvalho Chehab 499ae4a0502SMauro Carvalho ChehabLIBNDCTL: namespace enumeration example 500ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 501ae4a0502SMauro Carvalho ChehabNamespaces are indexed relative to their parent region, example below. 502ae4a0502SMauro Carvalho ChehabThese indexes are mostly static from boot to boot, but subsystem makes 503ae4a0502SMauro Carvalho Chehabno guarantees in this regard. For a static namespace identifier use its 504ae4a0502SMauro Carvalho Chehab'uuid' attribute. 505ae4a0502SMauro Carvalho Chehab 506ae4a0502SMauro Carvalho Chehab:: 507ae4a0502SMauro Carvalho Chehab 508ae4a0502SMauro Carvalho Chehab static struct ndctl_namespace 509ae4a0502SMauro Carvalho Chehab *get_namespace_by_id(struct ndctl_region *region, unsigned int id) 510ae4a0502SMauro Carvalho Chehab { 511ae4a0502SMauro Carvalho Chehab struct ndctl_namespace *ndns; 512ae4a0502SMauro Carvalho Chehab 513ae4a0502SMauro Carvalho Chehab ndctl_namespace_foreach(region, ndns) 514ae4a0502SMauro Carvalho Chehab if (ndctl_namespace_get_id(ndns) == id) 515ae4a0502SMauro Carvalho Chehab return ndns; 516ae4a0502SMauro Carvalho Chehab 517ae4a0502SMauro Carvalho Chehab return NULL; 518ae4a0502SMauro Carvalho Chehab } 519ae4a0502SMauro Carvalho Chehab 520ae4a0502SMauro Carvalho ChehabLIBNDCTL: namespace creation example 521ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 522ae4a0502SMauro Carvalho Chehab 523ae4a0502SMauro Carvalho ChehabIdle namespaces are automatically created by the kernel if a given 524ae4a0502SMauro Carvalho Chehabregion has enough available capacity to create a new namespace. 525ae4a0502SMauro Carvalho ChehabNamespace instantiation involves finding an idle namespace and 526ae4a0502SMauro Carvalho Chehabconfiguring it. For the most part the setting of namespace attributes 527ae4a0502SMauro Carvalho Chehabcan occur in any order, the only constraint is that 'uuid' must be set 528ae4a0502SMauro Carvalho Chehabbefore 'size'. This enables the kernel to track DPA allocations 529ae4a0502SMauro Carvalho Chehabinternally with a static identifier:: 530ae4a0502SMauro Carvalho Chehab 531ae4a0502SMauro Carvalho Chehab static int configure_namespace(struct ndctl_region *region, 532ae4a0502SMauro Carvalho Chehab struct ndctl_namespace *ndns, 533ae4a0502SMauro Carvalho Chehab struct namespace_parameters *parameters) 534ae4a0502SMauro Carvalho Chehab { 535ae4a0502SMauro Carvalho Chehab char devname[50]; 536ae4a0502SMauro Carvalho Chehab 537ae4a0502SMauro Carvalho Chehab snprintf(devname, sizeof(devname), "namespace%d.%d", 538ae4a0502SMauro Carvalho Chehab ndctl_region_get_id(region), paramaters->id); 539ae4a0502SMauro Carvalho Chehab 540ae4a0502SMauro Carvalho Chehab ndctl_namespace_set_alt_name(ndns, devname); 541ae4a0502SMauro Carvalho Chehab /* 'uuid' must be set prior to setting size! */ 542ae4a0502SMauro Carvalho Chehab ndctl_namespace_set_uuid(ndns, paramaters->uuid); 543ae4a0502SMauro Carvalho Chehab ndctl_namespace_set_size(ndns, paramaters->size); 544ae4a0502SMauro Carvalho Chehab /* unlike pmem namespaces, blk namespaces have a sector size */ 545ae4a0502SMauro Carvalho Chehab if (parameters->lbasize) 546ae4a0502SMauro Carvalho Chehab ndctl_namespace_set_sector_size(ndns, parameters->lbasize); 547ae4a0502SMauro Carvalho Chehab ndctl_namespace_enable(ndns); 548ae4a0502SMauro Carvalho Chehab } 549ae4a0502SMauro Carvalho Chehab 550ae4a0502SMauro Carvalho Chehab 551ae4a0502SMauro Carvalho ChehabWhy the Term "namespace"? 552ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^ 553ae4a0502SMauro Carvalho Chehab 554ae4a0502SMauro Carvalho Chehab 1. Why not "volume" for instance? "volume" ran the risk of confusing 555ae4a0502SMauro Carvalho Chehab ND (libnvdimm subsystem) to a volume manager like device-mapper. 556ae4a0502SMauro Carvalho Chehab 557ae4a0502SMauro Carvalho Chehab 2. The term originated to describe the sub-devices that can be created 558ae4a0502SMauro Carvalho Chehab within a NVME controller (see the nvme specification: 559d3603f4cSAlexander A. Klimov https://www.nvmexpress.org/specifications/), and NFIT namespaces are 560ae4a0502SMauro Carvalho Chehab meant to parallel the capabilities and configurability of 561ae4a0502SMauro Carvalho Chehab NVME-namespaces. 562ae4a0502SMauro Carvalho Chehab 563ae4a0502SMauro Carvalho Chehab 564ae4a0502SMauro Carvalho ChehabLIBNVDIMM/LIBNDCTL: Block Translation Table "btt" 565ae4a0502SMauro Carvalho Chehab------------------------------------------------- 566ae4a0502SMauro Carvalho Chehab 567f8669f1dSDan WilliamsA BTT (design document: https://pmem.io/2014/09/23/btt.html) is a 568f8669f1dSDan Williamspersonality driver for a namespace that fronts entire namespace as an 569f8669f1dSDan Williams'address abstraction'. 570ae4a0502SMauro Carvalho Chehab 571ae4a0502SMauro Carvalho ChehabLIBNVDIMM: btt layout 572ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^ 573ae4a0502SMauro Carvalho Chehab 574ae4a0502SMauro Carvalho ChehabEvery region will start out with at least one BTT device which is the 575ae4a0502SMauro Carvalho Chehabseed device. To activate it set the "namespace", "uuid", and 576ae4a0502SMauro Carvalho Chehab"sector_size" attributes and then bind the device to the nd_pmem or 577ae4a0502SMauro Carvalho Chehabnd_blk driver depending on the region type:: 578ae4a0502SMauro Carvalho Chehab 579ae4a0502SMauro Carvalho Chehab /sys/devices/platform/nfit_test.1/ndbus0/region0/btt0/ 580ae4a0502SMauro Carvalho Chehab |-- namespace 581ae4a0502SMauro Carvalho Chehab |-- delete 582ae4a0502SMauro Carvalho Chehab |-- devtype 583ae4a0502SMauro Carvalho Chehab |-- modalias 584ae4a0502SMauro Carvalho Chehab |-- numa_node 585ae4a0502SMauro Carvalho Chehab |-- sector_size 586ae4a0502SMauro Carvalho Chehab |-- subsystem -> ../../../../../bus/nd 587ae4a0502SMauro Carvalho Chehab |-- uevent 588ae4a0502SMauro Carvalho Chehab `-- uuid 589ae4a0502SMauro Carvalho Chehab 590ae4a0502SMauro Carvalho ChehabLIBNDCTL: btt creation example 591ae4a0502SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 592ae4a0502SMauro Carvalho Chehab 593ae4a0502SMauro Carvalho ChehabSimilar to namespaces an idle BTT device is automatically created per 594ae4a0502SMauro Carvalho Chehabregion. Each time this "seed" btt device is configured and enabled a new 595ae4a0502SMauro Carvalho Chehabseed is created. Creating a BTT configuration involves two steps of 596f8669f1dSDan Williamsfinding and idle BTT and assigning it to consume a namespace. 597f8669f1dSDan Williams 598f8669f1dSDan Williams:: 599ae4a0502SMauro Carvalho Chehab 600ae4a0502SMauro Carvalho Chehab static struct ndctl_btt *get_idle_btt(struct ndctl_region *region) 601ae4a0502SMauro Carvalho Chehab { 602ae4a0502SMauro Carvalho Chehab struct ndctl_btt *btt; 603ae4a0502SMauro Carvalho Chehab 604ae4a0502SMauro Carvalho Chehab ndctl_btt_foreach(region, btt) 605ae4a0502SMauro Carvalho Chehab if (!ndctl_btt_is_enabled(btt) 606ae4a0502SMauro Carvalho Chehab && !ndctl_btt_is_configured(btt)) 607ae4a0502SMauro Carvalho Chehab return btt; 608ae4a0502SMauro Carvalho Chehab 609ae4a0502SMauro Carvalho Chehab return NULL; 610ae4a0502SMauro Carvalho Chehab } 611ae4a0502SMauro Carvalho Chehab 612ae4a0502SMauro Carvalho Chehab static int configure_btt(struct ndctl_region *region, 613ae4a0502SMauro Carvalho Chehab struct btt_parameters *parameters) 614ae4a0502SMauro Carvalho Chehab { 615ae4a0502SMauro Carvalho Chehab btt = get_idle_btt(region); 616ae4a0502SMauro Carvalho Chehab 617ae4a0502SMauro Carvalho Chehab ndctl_btt_set_uuid(btt, parameters->uuid); 618ae4a0502SMauro Carvalho Chehab ndctl_btt_set_sector_size(btt, parameters->sector_size); 619ae4a0502SMauro Carvalho Chehab ndctl_btt_set_namespace(btt, parameters->ndns); 620ae4a0502SMauro Carvalho Chehab /* turn off raw mode device */ 621ae4a0502SMauro Carvalho Chehab ndctl_namespace_disable(parameters->ndns); 622ae4a0502SMauro Carvalho Chehab /* turn on btt access */ 623ae4a0502SMauro Carvalho Chehab ndctl_btt_enable(btt); 624ae4a0502SMauro Carvalho Chehab } 625ae4a0502SMauro Carvalho Chehab 626ae4a0502SMauro Carvalho ChehabOnce instantiated a new inactive btt seed device will appear underneath 627ae4a0502SMauro Carvalho Chehabthe region. 628ae4a0502SMauro Carvalho Chehab 629ae4a0502SMauro Carvalho ChehabOnce a "namespace" is removed from a BTT that instance of the BTT device 630ae4a0502SMauro Carvalho Chehabwill be deleted or otherwise reset to default values. This deletion is 631ae4a0502SMauro Carvalho Chehabonly at the device model level. In order to destroy a BTT the "info 632ae4a0502SMauro Carvalho Chehabblock" needs to be destroyed. Note, that to destroy a BTT the media 633ae4a0502SMauro Carvalho Chehabneeds to be written in raw mode. By default, the kernel will autodetect 634ae4a0502SMauro Carvalho Chehabthe presence of a BTT and disable raw mode. This autodetect behavior 635ae4a0502SMauro Carvalho Chehabcan be suppressed by enabling raw mode for the namespace via the 636ae4a0502SMauro Carvalho Chehabndctl_namespace_set_raw_mode() API. 637ae4a0502SMauro Carvalho Chehab 638ae4a0502SMauro Carvalho Chehab 639ae4a0502SMauro Carvalho ChehabSummary LIBNDCTL Diagram 640ae4a0502SMauro Carvalho Chehab------------------------ 641ae4a0502SMauro Carvalho Chehab 642ae4a0502SMauro Carvalho ChehabFor the given example above, here is the view of the objects as seen by the 643ae4a0502SMauro Carvalho ChehabLIBNDCTL API:: 644ae4a0502SMauro Carvalho Chehab 645ae4a0502SMauro Carvalho Chehab +---+ 646f8669f1dSDan Williams |CTX| 647f8669f1dSDan Williams +-+-+ 648f8669f1dSDan Williams | 649f8669f1dSDan Williams +-------+ | 650f8669f1dSDan Williams | DIMM0 <-+ | +---------+ +--------------+ +---------------+ 651f8669f1dSDan Williams +-------+ | | +-> REGION0 +---> NAMESPACE0.0 +--> PMEM8 "pm0.0" | 652ae4a0502SMauro Carvalho Chehab | DIMM1 <-+ +-v--+ | +---------+ +--------------+ +---------------+ 653f8669f1dSDan Williams +-------+ +-+BUS0+-| +---------+ +--------------+ +----------------------+ 654f8669f1dSDan Williams | DIMM2 <-+ +----+ +-> REGION1 +---> NAMESPACE1.0 +--> PMEM6 "pm1.0" | BTT1 | 655f8669f1dSDan Williams +-------+ | | +---------+ +--------------+ +---------------+------+ 656f8669f1dSDan Williams | DIMM3 <-+ 657f8669f1dSDan Williams +-------+ 658