1*13bccf87SKishon Vijay Abraham I.. SPDX-License-Identifier: GPL-2.0 2*13bccf87SKishon Vijay Abraham I 3*13bccf87SKishon Vijay Abraham I================= 4*13bccf87SKishon Vijay Abraham IPCI NTB Function 5*13bccf87SKishon Vijay Abraham I================= 6*13bccf87SKishon Vijay Abraham I 7*13bccf87SKishon Vijay Abraham I:Author: Kishon Vijay Abraham I <kishon@ti.com> 8*13bccf87SKishon Vijay Abraham I 9*13bccf87SKishon Vijay Abraham IPCI Non-Transparent Bridges (NTB) allow two host systems to communicate 10*13bccf87SKishon Vijay Abraham Iwith each other by exposing each host as a device to the other host. 11*13bccf87SKishon Vijay Abraham INTBs typically support the ability to generate interrupts on the remote 12*13bccf87SKishon Vijay Abraham Imachine, expose memory ranges as BARs, and perform DMA. They also support 13*13bccf87SKishon Vijay Abraham Iscratchpads, which are areas of memory within the NTB that are accessible 14*13bccf87SKishon Vijay Abraham Ifrom both machines. 15*13bccf87SKishon Vijay Abraham I 16*13bccf87SKishon Vijay Abraham IPCI NTB Function allows two different systems (or hosts) to communicate 17*13bccf87SKishon Vijay Abraham Iwith each other by configuring the endpoint instances in such a way that 18*13bccf87SKishon Vijay Abraham Itransactions from one system are routed to the other system. 19*13bccf87SKishon Vijay Abraham I 20*13bccf87SKishon Vijay Abraham IIn the below diagram, PCI NTB function configures the SoC with multiple 21*13bccf87SKishon Vijay Abraham IPCI Endpoint (EP) instances in such a way that transactions from one EP 22*13bccf87SKishon Vijay Abraham Icontroller are routed to the other EP controller. Once PCI NTB function 23*13bccf87SKishon Vijay Abraham Iconfigures the SoC with multiple EP instances, HOST1 and HOST2 can 24*13bccf87SKishon Vijay Abraham Icommunicate with each other using SoC as a bridge. 25*13bccf87SKishon Vijay Abraham I 26*13bccf87SKishon Vijay Abraham I.. code-block:: text 27*13bccf87SKishon Vijay Abraham I 28*13bccf87SKishon Vijay Abraham I +-------------+ +-------------+ 29*13bccf87SKishon Vijay Abraham I | | | | 30*13bccf87SKishon Vijay Abraham I | HOST1 | | HOST2 | 31*13bccf87SKishon Vijay Abraham I | | | | 32*13bccf87SKishon Vijay Abraham I +------^------+ +------^------+ 33*13bccf87SKishon Vijay Abraham I | | 34*13bccf87SKishon Vijay Abraham I | | 35*13bccf87SKishon Vijay Abraham I +---------|-------------------------------------------------|---------+ 36*13bccf87SKishon Vijay Abraham I | +------v------+ +------v------+ | 37*13bccf87SKishon Vijay Abraham I | | | | | | 38*13bccf87SKishon Vijay Abraham I | | EP | | EP | | 39*13bccf87SKishon Vijay Abraham I | | CONTROLLER1 | | CONTROLLER2 | | 40*13bccf87SKishon Vijay Abraham I | | <-----------------------------------> | | 41*13bccf87SKishon Vijay Abraham I | | | | | | 42*13bccf87SKishon Vijay Abraham I | | | | | | 43*13bccf87SKishon Vijay Abraham I | | | SoC With Multiple EP Instances | | | 44*13bccf87SKishon Vijay Abraham I | | | (Configured using NTB Function) | | | 45*13bccf87SKishon Vijay Abraham I | +-------------+ +-------------+ | 46*13bccf87SKishon Vijay Abraham I +---------------------------------------------------------------------+ 47*13bccf87SKishon Vijay Abraham I 48*13bccf87SKishon Vijay Abraham IConstructs used for Implementing NTB 49*13bccf87SKishon Vijay Abraham I==================================== 50*13bccf87SKishon Vijay Abraham I 51*13bccf87SKishon Vijay Abraham I 1) Config Region 52*13bccf87SKishon Vijay Abraham I 2) Self Scratchpad Registers 53*13bccf87SKishon Vijay Abraham I 3) Peer Scratchpad Registers 54*13bccf87SKishon Vijay Abraham I 4) Doorbell (DB) Registers 55*13bccf87SKishon Vijay Abraham I 5) Memory Window (MW) 56*13bccf87SKishon Vijay Abraham I 57*13bccf87SKishon Vijay Abraham I 58*13bccf87SKishon Vijay Abraham IConfig Region: 59*13bccf87SKishon Vijay Abraham I-------------- 60*13bccf87SKishon Vijay Abraham I 61*13bccf87SKishon Vijay Abraham IConfig Region is a construct that is specific to NTB implemented using NTB 62*13bccf87SKishon Vijay Abraham IEndpoint Function Driver. The host and endpoint side NTB function driver will 63*13bccf87SKishon Vijay Abraham Iexchange information with each other using this region. Config Region has 64*13bccf87SKishon Vijay Abraham IControl/Status Registers for configuring the Endpoint Controller. Host can 65*13bccf87SKishon Vijay Abraham Iwrite into this region for configuring the outbound Address Translation Unit 66*13bccf87SKishon Vijay Abraham I(ATU) and to indicate the link status. Endpoint can indicate the status of 67*13bccf87SKishon Vijay Abraham Icommands issued by host in this region. Endpoint can also indicate the 68*13bccf87SKishon Vijay Abraham Iscratchpad offset and number of memory windows to the host using this region. 69*13bccf87SKishon Vijay Abraham I 70*13bccf87SKishon Vijay Abraham IThe format of Config Region is given below. All the fields here are 32 bits. 71*13bccf87SKishon Vijay Abraham I 72*13bccf87SKishon Vijay Abraham I.. code-block:: text 73*13bccf87SKishon Vijay Abraham I 74*13bccf87SKishon Vijay Abraham I +------------------------+ 75*13bccf87SKishon Vijay Abraham I | COMMAND | 76*13bccf87SKishon Vijay Abraham I +------------------------+ 77*13bccf87SKishon Vijay Abraham I | ARGUMENT | 78*13bccf87SKishon Vijay Abraham I +------------------------+ 79*13bccf87SKishon Vijay Abraham I | STATUS | 80*13bccf87SKishon Vijay Abraham I +------------------------+ 81*13bccf87SKishon Vijay Abraham I | TOPOLOGY | 82*13bccf87SKishon Vijay Abraham I +------------------------+ 83*13bccf87SKishon Vijay Abraham I | ADDRESS (LOWER 32) | 84*13bccf87SKishon Vijay Abraham I +------------------------+ 85*13bccf87SKishon Vijay Abraham I | ADDRESS (UPPER 32) | 86*13bccf87SKishon Vijay Abraham I +------------------------+ 87*13bccf87SKishon Vijay Abraham I | SIZE | 88*13bccf87SKishon Vijay Abraham I +------------------------+ 89*13bccf87SKishon Vijay Abraham I | NO OF MEMORY WINDOW | 90*13bccf87SKishon Vijay Abraham I +------------------------+ 91*13bccf87SKishon Vijay Abraham I | MEMORY WINDOW1 OFFSET | 92*13bccf87SKishon Vijay Abraham I +------------------------+ 93*13bccf87SKishon Vijay Abraham I | SPAD OFFSET | 94*13bccf87SKishon Vijay Abraham I +------------------------+ 95*13bccf87SKishon Vijay Abraham I | SPAD COUNT | 96*13bccf87SKishon Vijay Abraham I +------------------------+ 97*13bccf87SKishon Vijay Abraham I | DB ENTRY SIZE | 98*13bccf87SKishon Vijay Abraham I +------------------------+ 99*13bccf87SKishon Vijay Abraham I | DB DATA | 100*13bccf87SKishon Vijay Abraham I +------------------------+ 101*13bccf87SKishon Vijay Abraham I | : | 102*13bccf87SKishon Vijay Abraham I +------------------------+ 103*13bccf87SKishon Vijay Abraham I | : | 104*13bccf87SKishon Vijay Abraham I +------------------------+ 105*13bccf87SKishon Vijay Abraham I | DB DATA | 106*13bccf87SKishon Vijay Abraham I +------------------------+ 107*13bccf87SKishon Vijay Abraham I 108*13bccf87SKishon Vijay Abraham I 109*13bccf87SKishon Vijay Abraham I COMMAND: 110*13bccf87SKishon Vijay Abraham I 111*13bccf87SKishon Vijay Abraham I NTB function supports three commands: 112*13bccf87SKishon Vijay Abraham I 113*13bccf87SKishon Vijay Abraham I CMD_CONFIGURE_DOORBELL (0x1): Command to configure doorbell. Before 114*13bccf87SKishon Vijay Abraham I invoking this command, the host should allocate and initialize 115*13bccf87SKishon Vijay Abraham I MSI/MSI-X vectors (i.e., initialize the MSI/MSI-X Capability in the 116*13bccf87SKishon Vijay Abraham I Endpoint). The endpoint on receiving this command will configure 117*13bccf87SKishon Vijay Abraham I the outbound ATU such that transactions to Doorbell BAR will be routed 118*13bccf87SKishon Vijay Abraham I to the MSI/MSI-X address programmed by the host. The ARGUMENT 119*13bccf87SKishon Vijay Abraham I register should be populated with number of DBs to configure (in the 120*13bccf87SKishon Vijay Abraham I lower 16 bits) and if MSI or MSI-X should be configured (BIT 16). 121*13bccf87SKishon Vijay Abraham I 122*13bccf87SKishon Vijay Abraham I CMD_CONFIGURE_MW (0x2): Command to configure memory window (MW). The 123*13bccf87SKishon Vijay Abraham I host invokes this command after allocating a buffer that can be 124*13bccf87SKishon Vijay Abraham I accessed by remote host. The allocated address should be programmed 125*13bccf87SKishon Vijay Abraham I in the ADDRESS register (64 bit), the size should be programmed in 126*13bccf87SKishon Vijay Abraham I the SIZE register and the memory window index should be programmed 127*13bccf87SKishon Vijay Abraham I in the ARGUMENT register. The endpoint on receiving this command 128*13bccf87SKishon Vijay Abraham I will configure the outbound ATU such that transactions to MW BAR 129*13bccf87SKishon Vijay Abraham I are routed to the address provided by the host. 130*13bccf87SKishon Vijay Abraham I 131*13bccf87SKishon Vijay Abraham I CMD_LINK_UP (0x3): Command to indicate an NTB application is 132*13bccf87SKishon Vijay Abraham I bound to the EP device on the host side. Once the endpoint 133*13bccf87SKishon Vijay Abraham I receives this command from both the hosts, the endpoint will 134*13bccf87SKishon Vijay Abraham I raise a LINK_UP event to both the hosts to indicate the host 135*13bccf87SKishon Vijay Abraham I NTB applications can start communicating with each other. 136*13bccf87SKishon Vijay Abraham I 137*13bccf87SKishon Vijay Abraham I ARGUMENT: 138*13bccf87SKishon Vijay Abraham I 139*13bccf87SKishon Vijay Abraham I The value of this register is based on the commands issued in 140*13bccf87SKishon Vijay Abraham I command register. See COMMAND section for more information. 141*13bccf87SKishon Vijay Abraham I 142*13bccf87SKishon Vijay Abraham I TOPOLOGY: 143*13bccf87SKishon Vijay Abraham I 144*13bccf87SKishon Vijay Abraham I Set to NTB_TOPO_B2B_USD for Primary interface 145*13bccf87SKishon Vijay Abraham I Set to NTB_TOPO_B2B_DSD for Secondary interface 146*13bccf87SKishon Vijay Abraham I 147*13bccf87SKishon Vijay Abraham I ADDRESS/SIZE: 148*13bccf87SKishon Vijay Abraham I 149*13bccf87SKishon Vijay Abraham I Address and Size to be used while configuring the memory window. 150*13bccf87SKishon Vijay Abraham I See "CMD_CONFIGURE_MW" for more info. 151*13bccf87SKishon Vijay Abraham I 152*13bccf87SKishon Vijay Abraham I MEMORY WINDOW1 OFFSET: 153*13bccf87SKishon Vijay Abraham I 154*13bccf87SKishon Vijay Abraham I Memory Window 1 and Doorbell registers are packed together in the 155*13bccf87SKishon Vijay Abraham I same BAR. The initial portion of the region will have doorbell 156*13bccf87SKishon Vijay Abraham I registers and the latter portion of the region is for memory window 1. 157*13bccf87SKishon Vijay Abraham I This register will specify the offset of the memory window 1. 158*13bccf87SKishon Vijay Abraham I 159*13bccf87SKishon Vijay Abraham I NO OF MEMORY WINDOW: 160*13bccf87SKishon Vijay Abraham I 161*13bccf87SKishon Vijay Abraham I Specifies the number of memory windows supported by the NTB device. 162*13bccf87SKishon Vijay Abraham I 163*13bccf87SKishon Vijay Abraham I SPAD OFFSET: 164*13bccf87SKishon Vijay Abraham I 165*13bccf87SKishon Vijay Abraham I Self scratchpad region and config region are packed together in the 166*13bccf87SKishon Vijay Abraham I same BAR. The initial portion of the region will have config region 167*13bccf87SKishon Vijay Abraham I and the latter portion of the region is for self scratchpad. This 168*13bccf87SKishon Vijay Abraham I register will specify the offset of the self scratchpad registers. 169*13bccf87SKishon Vijay Abraham I 170*13bccf87SKishon Vijay Abraham I SPAD COUNT: 171*13bccf87SKishon Vijay Abraham I 172*13bccf87SKishon Vijay Abraham I Specifies the number of scratchpad registers supported by the NTB 173*13bccf87SKishon Vijay Abraham I device. 174*13bccf87SKishon Vijay Abraham I 175*13bccf87SKishon Vijay Abraham I DB ENTRY SIZE: 176*13bccf87SKishon Vijay Abraham I 177*13bccf87SKishon Vijay Abraham I Used to determine the offset within the DB BAR that should be written 178*13bccf87SKishon Vijay Abraham I in order to raise doorbell. EPF NTB can use either MSI or MSI-X to 179*13bccf87SKishon Vijay Abraham I ring doorbell (MSI-X support will be added later). MSI uses same 180*13bccf87SKishon Vijay Abraham I address for all the interrupts and MSI-X can provide different 181*13bccf87SKishon Vijay Abraham I addresses for different interrupts. The MSI/MSI-X address is provided 182*13bccf87SKishon Vijay Abraham I by the host and the address it gives is based on the MSI/MSI-X 183*13bccf87SKishon Vijay Abraham I implementation supported by the host. For instance, ARM platform 184*13bccf87SKishon Vijay Abraham I using GIC ITS will have the same MSI-X address for all the interrupts. 185*13bccf87SKishon Vijay Abraham I In order to support all the combinations and use the same mechanism 186*13bccf87SKishon Vijay Abraham I for both MSI and MSI-X, EPF NTB allocates a separate region in the 187*13bccf87SKishon Vijay Abraham I Outbound Address Space for each of the interrupts. This region will 188*13bccf87SKishon Vijay Abraham I be mapped to the MSI/MSI-X address provided by the host. If a host 189*13bccf87SKishon Vijay Abraham I provides the same address for all the interrupts, all the regions 190*13bccf87SKishon Vijay Abraham I will be translated to the same address. If a host provides different 191*13bccf87SKishon Vijay Abraham I addresses, the regions will be translated to different addresses. This 192*13bccf87SKishon Vijay Abraham I will ensure there is no difference while raising the doorbell. 193*13bccf87SKishon Vijay Abraham I 194*13bccf87SKishon Vijay Abraham I DB DATA: 195*13bccf87SKishon Vijay Abraham I 196*13bccf87SKishon Vijay Abraham I EPF NTB supports 32 interrupts, so there are 32 DB DATA registers. 197*13bccf87SKishon Vijay Abraham I This holds the MSI/MSI-X data that has to be written to MSI address 198*13bccf87SKishon Vijay Abraham I for raising doorbell interrupt. This will be populated by EPF NTB 199*13bccf87SKishon Vijay Abraham I while invoking CMD_CONFIGURE_DOORBELL. 200*13bccf87SKishon Vijay Abraham I 201*13bccf87SKishon Vijay Abraham IScratchpad Registers: 202*13bccf87SKishon Vijay Abraham I--------------------- 203*13bccf87SKishon Vijay Abraham I 204*13bccf87SKishon Vijay Abraham I Each host has its own register space allocated in the memory of NTB endpoint 205*13bccf87SKishon Vijay Abraham I controller. They are both readable and writable from both sides of the bridge. 206*13bccf87SKishon Vijay Abraham I They are used by applications built over NTB and can be used to pass control 207*13bccf87SKishon Vijay Abraham I and status information between both sides of a device. 208*13bccf87SKishon Vijay Abraham I 209*13bccf87SKishon Vijay Abraham I Scratchpad registers has 2 parts 210*13bccf87SKishon Vijay Abraham I 1) Self Scratchpad: Host's own register space 211*13bccf87SKishon Vijay Abraham I 2) Peer Scratchpad: Remote host's register space. 212*13bccf87SKishon Vijay Abraham I 213*13bccf87SKishon Vijay Abraham IDoorbell Registers: 214*13bccf87SKishon Vijay Abraham I------------------- 215*13bccf87SKishon Vijay Abraham I 216*13bccf87SKishon Vijay Abraham I Doorbell Registers are used by the hosts to interrupt each other. 217*13bccf87SKishon Vijay Abraham I 218*13bccf87SKishon Vijay Abraham IMemory Window: 219*13bccf87SKishon Vijay Abraham I-------------- 220*13bccf87SKishon Vijay Abraham I 221*13bccf87SKishon Vijay Abraham I Actual transfer of data between the two hosts will happen using the 222*13bccf87SKishon Vijay Abraham I memory window. 223*13bccf87SKishon Vijay Abraham I 224*13bccf87SKishon Vijay Abraham IModeling Constructs: 225*13bccf87SKishon Vijay Abraham I==================== 226*13bccf87SKishon Vijay Abraham I 227*13bccf87SKishon Vijay Abraham IThere are 5 or more distinct regions (config, self scratchpad, peer 228*13bccf87SKishon Vijay Abraham Iscratchpad, doorbell, one or more memory windows) to be modeled to achieve 229*13bccf87SKishon Vijay Abraham INTB functionality. At least one memory window is required while more than 230*13bccf87SKishon Vijay Abraham Ione is permitted. All these regions should be mapped to BARs for hosts to 231*13bccf87SKishon Vijay Abraham Iaccess these regions. 232*13bccf87SKishon Vijay Abraham I 233*13bccf87SKishon Vijay Abraham IIf one 32-bit BAR is allocated for each of these regions, the scheme would 234*13bccf87SKishon Vijay Abraham Ilook like this: 235*13bccf87SKishon Vijay Abraham I 236*13bccf87SKishon Vijay Abraham I====== =============== 237*13bccf87SKishon Vijay Abraham IBAR NO CONSTRUCTS USED 238*13bccf87SKishon Vijay Abraham I====== =============== 239*13bccf87SKishon Vijay Abraham IBAR0 Config Region 240*13bccf87SKishon Vijay Abraham IBAR1 Self Scratchpad 241*13bccf87SKishon Vijay Abraham IBAR2 Peer Scratchpad 242*13bccf87SKishon Vijay Abraham IBAR3 Doorbell 243*13bccf87SKishon Vijay Abraham IBAR4 Memory Window 1 244*13bccf87SKishon Vijay Abraham IBAR5 Memory Window 2 245*13bccf87SKishon Vijay Abraham I====== =============== 246*13bccf87SKishon Vijay Abraham I 247*13bccf87SKishon Vijay Abraham IHowever if we allocate a separate BAR for each of the regions, there would not 248*13bccf87SKishon Vijay Abraham Ibe enough BARs for all the regions in a platform that supports only 64-bit 249*13bccf87SKishon Vijay Abraham IBARs. 250*13bccf87SKishon Vijay Abraham I 251*13bccf87SKishon Vijay Abraham IIn order to be supported by most of the platforms, the regions should be 252*13bccf87SKishon Vijay Abraham Ipacked and mapped to BARs in a way that provides NTB functionality and 253*13bccf87SKishon Vijay Abraham Ialso makes sure the host doesn't access any region that it is not supposed 254*13bccf87SKishon Vijay Abraham Ito. 255*13bccf87SKishon Vijay Abraham I 256*13bccf87SKishon Vijay Abraham IThe following scheme is used in EPF NTB Function: 257*13bccf87SKishon Vijay Abraham I 258*13bccf87SKishon Vijay Abraham I====== =============================== 259*13bccf87SKishon Vijay Abraham IBAR NO CONSTRUCTS USED 260*13bccf87SKishon Vijay Abraham I====== =============================== 261*13bccf87SKishon Vijay Abraham IBAR0 Config Region + Self Scratchpad 262*13bccf87SKishon Vijay Abraham IBAR1 Peer Scratchpad 263*13bccf87SKishon Vijay Abraham IBAR2 Doorbell + Memory Window 1 264*13bccf87SKishon Vijay Abraham IBAR3 Memory Window 2 265*13bccf87SKishon Vijay Abraham IBAR4 Memory Window 3 266*13bccf87SKishon Vijay Abraham IBAR5 Memory Window 4 267*13bccf87SKishon Vijay Abraham I====== =============================== 268*13bccf87SKishon Vijay Abraham I 269*13bccf87SKishon Vijay Abraham IWith this scheme, for the basic NTB functionality 3 BARs should be sufficient. 270*13bccf87SKishon Vijay Abraham I 271*13bccf87SKishon Vijay Abraham IModeling Config/Scratchpad Region: 272*13bccf87SKishon Vijay Abraham I---------------------------------- 273*13bccf87SKishon Vijay Abraham I 274*13bccf87SKishon Vijay Abraham I.. code-block:: text 275*13bccf87SKishon Vijay Abraham I 276*13bccf87SKishon Vijay Abraham I +-----------------+------->+------------------+ +-----------------+ 277*13bccf87SKishon Vijay Abraham I | BAR0 | | CONFIG REGION | | BAR0 | 278*13bccf87SKishon Vijay Abraham I +-----------------+----+ +------------------+<-------+-----------------+ 279*13bccf87SKishon Vijay Abraham I | BAR1 | | |SCRATCHPAD REGION | | BAR1 | 280*13bccf87SKishon Vijay Abraham I +-----------------+ +-->+------------------+<-------+-----------------+ 281*13bccf87SKishon Vijay Abraham I | BAR2 | Local Memory | BAR2 | 282*13bccf87SKishon Vijay Abraham I +-----------------+ +-----------------+ 283*13bccf87SKishon Vijay Abraham I | BAR3 | | BAR3 | 284*13bccf87SKishon Vijay Abraham I +-----------------+ +-----------------+ 285*13bccf87SKishon Vijay Abraham I | BAR4 | | BAR4 | 286*13bccf87SKishon Vijay Abraham I +-----------------+ +-----------------+ 287*13bccf87SKishon Vijay Abraham I | BAR5 | | BAR5 | 288*13bccf87SKishon Vijay Abraham I +-----------------+ +-----------------+ 289*13bccf87SKishon Vijay Abraham I EP CONTROLLER 1 EP CONTROLLER 2 290*13bccf87SKishon Vijay Abraham I 291*13bccf87SKishon Vijay Abraham IAbove diagram shows Config region + Scratchpad region for HOST1 (connected to 292*13bccf87SKishon Vijay Abraham IEP controller 1) allocated in local memory. The HOST1 can access the config 293*13bccf87SKishon Vijay Abraham Iregion and scratchpad region (self scratchpad) using BAR0 of EP controller 1. 294*13bccf87SKishon Vijay Abraham IThe peer host (HOST2 connected to EP controller 2) can also access this 295*13bccf87SKishon Vijay Abraham Iscratchpad region (peer scratchpad) using BAR1 of EP controller 2. This 296*13bccf87SKishon Vijay Abraham Idiagram shows the case where Config region and Scratchpad regions are allocated 297*13bccf87SKishon Vijay Abraham Ifor HOST1, however the same is applicable for HOST2. 298*13bccf87SKishon Vijay Abraham I 299*13bccf87SKishon Vijay Abraham IModeling Doorbell/Memory Window 1: 300*13bccf87SKishon Vijay Abraham I---------------------------------- 301*13bccf87SKishon Vijay Abraham I 302*13bccf87SKishon Vijay Abraham I.. code-block:: text 303*13bccf87SKishon Vijay Abraham I 304*13bccf87SKishon Vijay Abraham I +-----------------+ +----->+----------------+-----------+-----------------+ 305*13bccf87SKishon Vijay Abraham I | BAR0 | | | Doorbell 1 +-----------> MSI-X ADDRESS 1 | 306*13bccf87SKishon Vijay Abraham I +-----------------+ | +----------------+ +-----------------+ 307*13bccf87SKishon Vijay Abraham I | BAR1 | | | Doorbell 2 +---------+ | | 308*13bccf87SKishon Vijay Abraham I +-----------------+----+ +----------------+ | | | 309*13bccf87SKishon Vijay Abraham I | BAR2 | | Doorbell 3 +-------+ | +-----------------+ 310*13bccf87SKishon Vijay Abraham I +-----------------+----+ +----------------+ | +-> MSI-X ADDRESS 2 | 311*13bccf87SKishon Vijay Abraham I | BAR3 | | | Doorbell 4 +-----+ | +-----------------+ 312*13bccf87SKishon Vijay Abraham I +-----------------+ | |----------------+ | | | | 313*13bccf87SKishon Vijay Abraham I | BAR4 | | | | | | +-----------------+ 314*13bccf87SKishon Vijay Abraham I +-----------------+ | | MW1 +---+ | +-->+ MSI-X ADDRESS 3|| 315*13bccf87SKishon Vijay Abraham I | BAR5 | | | | | | +-----------------+ 316*13bccf87SKishon Vijay Abraham I +-----------------+ +----->-----------------+ | | | | 317*13bccf87SKishon Vijay Abraham I EP CONTROLLER 1 | | | | +-----------------+ 318*13bccf87SKishon Vijay Abraham I | | | +---->+ MSI-X ADDRESS 4 | 319*13bccf87SKishon Vijay Abraham I +----------------+ | +-----------------+ 320*13bccf87SKishon Vijay Abraham I EP CONTROLLER 2 | | | 321*13bccf87SKishon Vijay Abraham I (OB SPACE) | | | 322*13bccf87SKishon Vijay Abraham I +-------> MW1 | 323*13bccf87SKishon Vijay Abraham I | | 324*13bccf87SKishon Vijay Abraham I | | 325*13bccf87SKishon Vijay Abraham I +-----------------+ 326*13bccf87SKishon Vijay Abraham I | | 327*13bccf87SKishon Vijay Abraham I | | 328*13bccf87SKishon Vijay Abraham I | | 329*13bccf87SKishon Vijay Abraham I | | 330*13bccf87SKishon Vijay Abraham I | | 331*13bccf87SKishon Vijay Abraham I +-----------------+ 332*13bccf87SKishon Vijay Abraham I PCI Address Space 333*13bccf87SKishon Vijay Abraham I (Managed by HOST2) 334*13bccf87SKishon Vijay Abraham I 335*13bccf87SKishon Vijay Abraham IAbove diagram shows how the doorbell and memory window 1 is mapped so that 336*13bccf87SKishon Vijay Abraham IHOST1 can raise doorbell interrupt on HOST2 and also how HOST1 can access 337*13bccf87SKishon Vijay Abraham Ibuffers exposed by HOST2 using memory window1 (MW1). Here doorbell and 338*13bccf87SKishon Vijay Abraham Imemory window 1 regions are allocated in EP controller 2 outbound (OB) address 339*13bccf87SKishon Vijay Abraham Ispace. Allocating and configuring BARs for doorbell and memory window1 340*13bccf87SKishon Vijay Abraham Iis done during the initialization phase of NTB endpoint function driver. 341*13bccf87SKishon Vijay Abraham IMapping from EP controller 2 OB space to PCI address space is done when HOST2 342*13bccf87SKishon Vijay Abraham Isends CMD_CONFIGURE_MW/CMD_CONFIGURE_DOORBELL. 343*13bccf87SKishon Vijay Abraham I 344*13bccf87SKishon Vijay Abraham IModeling Optional Memory Windows: 345*13bccf87SKishon Vijay Abraham I--------------------------------- 346*13bccf87SKishon Vijay Abraham I 347*13bccf87SKishon Vijay Abraham IThis is modeled the same was as MW1 but each of the additional memory windows 348*13bccf87SKishon Vijay Abraham Iis mapped to separate BARs. 349