1*c1800a16SAleksandr BezzubikovGeneric PCI Express to PCI Bridge 2*c1800a16SAleksandr Bezzubikov================================ 3*c1800a16SAleksandr Bezzubikov 4*c1800a16SAleksandr BezzubikovDescription 5*c1800a16SAleksandr Bezzubikov=========== 6*c1800a16SAleksandr BezzubikovPCIE-to-PCI bridge is a new method for legacy PCI 7*c1800a16SAleksandr Bezzubikovhierarchies creation on Q35 machines. 8*c1800a16SAleksandr Bezzubikov 9*c1800a16SAleksandr BezzubikovPreviously Intel DMI-to-PCI bridge was used for this purpose. 10*c1800a16SAleksandr BezzubikovBut due to its strict limitations - no support of hot-plug, 11*c1800a16SAleksandr Bezzubikovno cross-platform and cross-architecture support - a new generic 12*c1800a16SAleksandr BezzubikovPCIE-to-PCI bridge should now be used for any legacy PCI device usage 13*c1800a16SAleksandr Bezzubikovwith PCI Express machine. 14*c1800a16SAleksandr Bezzubikov 15*c1800a16SAleksandr BezzubikovThis generic PCIE-PCI bridge is a cross-platform device, 16*c1800a16SAleksandr Bezzubikovcan be hot-plugged into appropriate root port (requires additional actions, 17*c1800a16SAleksandr Bezzubikovsee 'PCIE-PCI bridge hot-plug' section), 18*c1800a16SAleksandr Bezzubikovand supports devices hot-plug into the bridge itself 19*c1800a16SAleksandr Bezzubikov(with some limitations, see below). 20*c1800a16SAleksandr Bezzubikov 21*c1800a16SAleksandr BezzubikovHot-plug of legacy PCI devices into the bridge 22*c1800a16SAleksandr Bezzubikovis provided by bridge's built-in Standard hot-plug Controller. 23*c1800a16SAleksandr BezzubikovThough it still has some limitations, see below. 24*c1800a16SAleksandr Bezzubikov 25*c1800a16SAleksandr BezzubikovPCIE-PCI bridge hot-plug 26*c1800a16SAleksandr Bezzubikov======================= 27*c1800a16SAleksandr BezzubikovGuest OSes require extra efforts to enable PCIE-PCI bridge hot-plug. 28*c1800a16SAleksandr BezzubikovMotivation - now on init any PCI Express root port which doesn't have 29*c1800a16SAleksandr Bezzubikovany device plugged in, has no free buses reserved to provide any of them 30*c1800a16SAleksandr Bezzubikovto a hot-plugged devices in future. 31*c1800a16SAleksandr Bezzubikov 32*c1800a16SAleksandr BezzubikovTo solve this problem we reserve additional buses on a firmware level. 33*c1800a16SAleksandr BezzubikovCurrently only SeaBIOS is supported. 34*c1800a16SAleksandr BezzubikovThe way of bus number to reserve delivery is special 35*c1800a16SAleksandr BezzubikovRed Hat vendor-specific PCI capability, added to the root port 36*c1800a16SAleksandr Bezzubikovthat is planned to have PCIE-PCI bridge hot-plugged in. 37*c1800a16SAleksandr Bezzubikov 38*c1800a16SAleksandr BezzubikovCapability layout (defined in include/hw/pci/pci_bridge.h): 39*c1800a16SAleksandr Bezzubikov 40*c1800a16SAleksandr Bezzubikov uint8_t id; Standard PCI capability header field 41*c1800a16SAleksandr Bezzubikov uint8_t next; Standard PCI capability header field 42*c1800a16SAleksandr Bezzubikov uint8_t len; Standard PCI vendor-specific capability header field 43*c1800a16SAleksandr Bezzubikov 44*c1800a16SAleksandr Bezzubikov uint8_t type; Red Hat vendor-specific capability type 45*c1800a16SAleksandr Bezzubikov List of currently existing types: 46*c1800a16SAleksandr Bezzubikov RESOURCE_RESERVE = 1 47*c1800a16SAleksandr Bezzubikov 48*c1800a16SAleksandr Bezzubikov 49*c1800a16SAleksandr Bezzubikov uint32_t bus_res; Minimum number of buses to reserve 50*c1800a16SAleksandr Bezzubikov 51*c1800a16SAleksandr Bezzubikov uint64_t io; IO space to reserve 52*c1800a16SAleksandr Bezzubikov uint32_t mem Non-prefetchable memory to reserve 53*c1800a16SAleksandr Bezzubikov 54*c1800a16SAleksandr Bezzubikov At most one of the following two fields may be set to a value 55*c1800a16SAleksandr Bezzubikov different from -1: 56*c1800a16SAleksandr Bezzubikov uint32_t mem_pref_32; Prefetchable memory to reserve (32-bit MMIO) 57*c1800a16SAleksandr Bezzubikov uint64_t mem_pref_64; Prefetchable memory to reserve (64-bit MMIO) 58*c1800a16SAleksandr Bezzubikov 59*c1800a16SAleksandr BezzubikovIf any reservation field is -1 then this kind of reservation is not 60*c1800a16SAleksandr Bezzubikovneeded and must be ignored by firmware. 61*c1800a16SAleksandr Bezzubikov 62*c1800a16SAleksandr BezzubikovAt the moment this capability is used only in QEMU generic PCIe root port 63*c1800a16SAleksandr Bezzubikov(-device pcie-root-port). Capability construction function takes all reservation 64*c1800a16SAleksandr Bezzubikovfields values from corresponding device properties. By default all of them are 65*c1800a16SAleksandr Bezzubikovset to -1 to leave root port's default behavior unchanged. 66*c1800a16SAleksandr Bezzubikov 67*c1800a16SAleksandr BezzubikovUsage 68*c1800a16SAleksandr Bezzubikov===== 69*c1800a16SAleksandr BezzubikovA detailed command line would be: 70*c1800a16SAleksandr Bezzubikov 71*c1800a16SAleksandr Bezzubikov[qemu-bin + storage options] \ 72*c1800a16SAleksandr Bezzubikov-m 2G \ 73*c1800a16SAleksandr Bezzubikov-device pcie-root-port,bus=pcie.0,id=rp1 \ 74*c1800a16SAleksandr Bezzubikov-device pcie-root-port,bus=pcie.0,id=rp2 \ 75*c1800a16SAleksandr Bezzubikov-device pcie-root-port,bus=pcie.0,id=rp3,bus-reserve=1 \ 76*c1800a16SAleksandr Bezzubikov-device pcie-pci-bridge,id=br1,bus=rp1 \ 77*c1800a16SAleksandr Bezzubikov-device pcie-pci-bridge,id=br2,bus=rp2 \ 78*c1800a16SAleksandr Bezzubikov-device e1000,bus=br1,addr=8 79*c1800a16SAleksandr Bezzubikov 80*c1800a16SAleksandr BezzubikovThen in monitor it's OK to execute next commands: 81*c1800a16SAleksandr Bezzubikovdevice_add pcie-pci-bridge,id=br3,bus=rp3 \ 82*c1800a16SAleksandr Bezzubikovdevice_add e1000,bus=br2,addr=1 \ 83*c1800a16SAleksandr Bezzubikovdevice_add e1000,bus=br3,addr=1 84*c1800a16SAleksandr Bezzubikov 85*c1800a16SAleksandr BezzubikovHere you have: 86*c1800a16SAleksandr Bezzubikov (1) Cold-plugged: 87*c1800a16SAleksandr Bezzubikov - Root ports: 1 QEMU generic root port with the capability mentioned above, 88*c1800a16SAleksandr Bezzubikov 2 QEMU generic root ports without this capability; 89*c1800a16SAleksandr Bezzubikov - 2 PCIE-PCI bridges plugged into 2 different root ports; 90*c1800a16SAleksandr Bezzubikov - e1000 plugged into the first bridge. 91*c1800a16SAleksandr Bezzubikov (2) Hot-plugged: 92*c1800a16SAleksandr Bezzubikov - PCIE-PCI bridge, plugged into QEMU generic root port; 93*c1800a16SAleksandr Bezzubikov - 2 e1000 cards, one plugged into the cold-plugged PCIE-PCI bridge, 94*c1800a16SAleksandr Bezzubikov another plugged into the hot-plugged bridge. 95*c1800a16SAleksandr Bezzubikov 96*c1800a16SAleksandr BezzubikovLimitations 97*c1800a16SAleksandr Bezzubikov=========== 98*c1800a16SAleksandr BezzubikovThe PCIE-PCI bridge can be hot-plugged only into pcie-root-port that 99*c1800a16SAleksandr Bezzubikovhas proper 'bus-reserve' property value to provide secondary bus for the 100*c1800a16SAleksandr Bezzubikovhot-plugged bridge. 101*c1800a16SAleksandr Bezzubikov 102*c1800a16SAleksandr BezzubikovWindows 7 and older versions don't support hot-plug devices into the PCIE-PCI bridge. 103*c1800a16SAleksandr BezzubikovTo enable device hot-plug into the bridge on Linux there're 3 ways: 104*c1800a16SAleksandr Bezzubikov1) Build shpchp module with this patch http://www.spinics.net/lists/linux-pci/msg63052.html 105*c1800a16SAleksandr Bezzubikov2) Use kernel 4.14+ where the patch mentioned above is already merged. 106*c1800a16SAleksandr Bezzubikov3) Set 'msi' property to off - this forces the bridge to use legacy INTx, 107*c1800a16SAleksandr Bezzubikov which allows the bridge to notify the OS about hot-plug event without having 108*c1800a16SAleksandr Bezzubikov BUSMASTER set. 109*c1800a16SAleksandr Bezzubikov 110*c1800a16SAleksandr BezzubikovImplementation 111*c1800a16SAleksandr Bezzubikov============== 112*c1800a16SAleksandr BezzubikovThe PCIE-PCI bridge is based on PCI-PCI bridge, but also accumulates PCI Express 113*c1800a16SAleksandr Bezzubikovfeatures as a PCI Express device (is_express=1). 114*c1800a16SAleksandr Bezzubikov 115