1--- 2layout: page 3title: fi_shm(7) 4tagline: Libfabric Programmer's Manual 5--- 6{% include JB/setup %} 7 8# NAME 9 10fi_shm \- The SHM Fabric Provider 11 12# OVERVIEW 13 14The SHM provider is a complete provider that can be used on Linux 15systems supporting shared memory and process_vm_readv/process_vm_writev 16calls. The provider is intended to provide high-performance communication 17between processes on the same system. 18 19# SUPPORTED FEATURES 20 21This release contains an initial implementation of the SHM provider that 22offers the following support: 23 24*Endpoint types* 25: The provider supports only endpoint type *FI_EP_RDM*. 26 27*Endpoint capabilities* 28: Endpoints cna support any combinations of the following data transfer 29capabilities: *FI_MSG*, *FI_TAGGED*, *FI_RMA*, amd *FI_ATOMICS*. These 30capabilities can be further defined by *FI_SEND*, *FI_RECV*, *FI_READ*, 31*FI_WRITE*, *FI_REMOTE_READ*, and *FI_REMOTE_WRITE* to limit the direction 32of operations. 33 34*Modes* 35: The provider does not require the use of any mode bits. 36 37*Progress* 38: The SHM provider supports *FI_PROGRESS_MANUAL*. Receive side data buffers are 39 not modified outside of completion processing routines. The provider processes 40 messages using three different methods, based on the size of the message. 41 For messages smaller than 4096 bytes, tx completions are generated immediately 42 after the send. For larger messages, tx completions are not generated until 43 the receiving side has processed the message. 44 45*Address Format* 46: The SHM provider uses the address format FI_ADDR_STR, which follows the general 47 format pattern "[prefix]://[addr]". The application can provide addresses 48 through the node or hints parameter. As long as the address is in a valid 49 FI_ADDR_STR format (contains "://"), the address will be used as is. If the 50 application input is incorrectly formatted or no input was provided, the SHM 51 provider will resolve it according to the following SHM provider standards: 52 53 (flags & FI_SOURCE) ? src_addr : dest_addr = 54 - if (node && service) : "fi_ns://node:service" 55 - if (service) : "fi_ns://service" 56 - if (node && !service) : "fi_shm://node" 57 - if (!node && !service) : "fi_shm://PID" 58 59 !(flags & FI_SOURCE) 60 - src_addr = "fi_shm://PID" 61 62 In other words, if the application provides a source and/or destination 63 address in an acceptable FI_ADDR_STR format (contains "://"), the call 64 to util_getinfo will successfully fill in src_addr and dest_addr with 65 the provided input. If the input is not in an ADDR_STR format, the 66 shared memory provider will then create a proper FI_ADDR_STR address 67 with either the "fi_ns://" (node/service format) or "fi_shm://" (shm format) 68 prefixes signaling whether the addr is a "unique" address and does or does 69 not need an extra endpoint name identifier appended in order to make it 70 unique. For the shared memory provider, we assume that the service 71 (with or without a node) is enough to make it unique, but a node alone is 72 not sufficient. If only a node is provided, the "fi_shm://" prefix is used 73 to signify that it is not a unique address. If no node or service are 74 provided (and in the case of setting the src address without FI_SOURCE and 75 no hints), the process ID will be used as a default address. 76 On endpoint creation, if the src_addr has the "fi_shm://" prefix, the provider 77 will append ":[uid]:[dom_idx]:[ep_idx]" as a unique endpoint name (essentially, 78 in place of a service). In the case of the "fi_ns://" prefix (or any other 79 prefix if one was provided by the application), no supplemental information 80 is required to make it unique and it will remain with only the 81 application-defined address. Note that the actual endpoint name will not 82 include the FI_ADDR_STR "*://" prefix since it cannot be included in any 83 shared memory region names. The provider will strip off the prefix before 84 setting the endpoint name. As a result, the addresses 85 "fi_prefix1://my_node:my_service" and "fi_prefix2://my_node:my_service" 86 would result in endpoints and regions of the same name. 87 The application can also override the endpoint name after creating an 88 endpoint using setname() without any address format restrictions. 89 90*Msg flags* 91 The provider currently only supports the FI_REMOTE_CQ_DATA msg flag. 92 93*MR registration mode* 94 The provider implements FI_MR_VIRT_ADDR memory mode. 95 96*Atomic operations* 97 The provider supports all combinations of datatype and operations as long 98 as the message is less than 4096 bytes (or 2048 for compare operations). 99 100# LIMITATIONS 101 102The SHM provider has hard-coded maximums for supported queue sizes and data 103transfers. These values are reflected in the related fabric attribute 104structures 105 106EPs must be bound to both RX and TX CQs. 107 108No support for counters. 109 110# RUNTIME PARAMETERS 111 112The *shm* provider checks for the following environment variables: 113 114*FI_SHM_SAR_THRESHOLD* 115: Maximum message size to use segmentation protocol before switching 116 to mmap (only valid when CMA is not available). Default: SIZE_MAX 117 (18446744073709551615) 118 119*FI_SHM_TX_SIZE* 120: Maximum number of outstanding tx operations. Default 1024 121 122*FI_SHM_RX_SIZE* 123: Maximum number of outstanding rx operations. Default 1024 124 125# SEE ALSO 126 127[`fabric`(7)](fabric.7.html), 128[`fi_provider`(7)](fi_provider.7.html), 129[`fi_getinfo`(3)](fi_getinfo.3.html) 130