1PowerNV family boards (``powernv8``, ``powernv9``) 2================================================================== 3 4PowerNV (as Non-Virtualized) is the "baremetal" platform using the 5OPAL firmware. It runs Linux on IBM and OpenPOWER systems and it can 6be used as an hypervisor OS, running KVM guests, or simply as a host 7OS. 8 9The PowerNV QEMU machine tries to emulate a PowerNV system at the 10level of the skiboot firmware, which loads the OS and provides some 11runtime services. Power Systems have a lower firmware (HostBoot) that 12does low level system initialization, like DRAM training. This is 13beyond the scope of what QEMU addresses today. 14 15Supported devices 16----------------- 17 18 * Multi processor support for POWER8, POWER8NVL and POWER9. 19 * XSCOM, serial communication sideband bus to configure chiplets 20 * Simple LPC Controller 21 * Processor Service Interface (PSI) Controller 22 * Interrupt Controller, XICS (POWER8) and XIVE (POWER9) 23 * POWER8 PHB3 PCIe Host bridge and POWER9 PHB4 PCIe Host bridge 24 * Simple OCC is an on-chip microcontroller used for power management 25 tasks 26 * iBT device to handle BMC communication, with the internal BMC 27 simulator provided by QEMU or an external BMC such as an Aspeed 28 QEMU machine. 29 * PNOR containing the different firmware partitions. 30 31Missing devices 32--------------- 33 34A lot is missing, among which : 35 36 * POWER10 processor 37 * XIVE2 (POWER10) interrupt controller 38 * I2C controllers (yet to be merged) 39 * NPU/NPU2/NPU3 controllers 40 * EEH support for PCIe Host bridge controllers 41 * NX controller 42 * VAS controller 43 * chipTOD (Time Of Day) 44 * Self Boot Engine (SBE). 45 * FSI bus 46 47Firmware 48-------- 49 50The OPAL firmware (OpenPower Abstraction Layer) for OpenPower systems 51includes the runtime services ``skiboot`` and the bootloader kernel and 52initramfs ``skiroot``. Source code can be found on GitHub: 53 54 https://github.com/open-power. 55 56Prebuilt images of ``skiboot`` and ``skiboot`` are made available on the `OpenPOWER <https://openpower.xyz/job/openpower/job/openpower-op-build/>`__ site. To boot a POWER9 machine, use the `witherspoon <https://openpower.xyz/job/openpower/job/openpower-op-build/label=slave,target=witherspoon/lastSuccessfulBuild/>`__ images. For POWER8, use 57the `palmetto <https://openpower.xyz/job/openpower/job/openpower-op-build/label=slave,target=palmetto/lastSuccessfulBuild/>`__ images. 58 59QEMU includes a prebuilt image of ``skiboot`` which is updated when a 60more recent version is required by the models. 61 62Boot options 63------------ 64 65Here is a simple setup with one e1000e NIC : 66 67.. code-block:: bash 68 69 $ qemu-system-ppc64 -m 2G -machine powernv9 -smp 2,cores=2,threads=1 \ 70 -accel tcg,thread=single \ 71 -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 \ 72 -netdev user,id=net0,hostfwd=::20022-:22,hostname=pnv \ 73 -kernel ./zImage.epapr \ 74 -initrd ./rootfs.cpio.xz \ 75 -nographic 76 77and a SATA disk : 78 79.. code-block:: bash 80 81 -device ich9-ahci,id=sata0,bus=pcie.1,addr=0x0 \ 82 -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive0,format=qcow2,cache=none \ 83 -device ide-hd,bus=sata0.0,unit=0,drive=drive0,id=ide,bootindex=1 \ 84 85Complex PCIe configuration 86~~~~~~~~~~~~~~~~~~~~~~~~~~ 87Six PHBs are defined per chip (POWER9) but no default PCI layout is 88provided (to be compatible with libvirt). One PCI device can be added 89on any of the available PCIe slots using command line options such as: 90 91.. code-block:: bash 92 93 -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 94 -netdev bridge,id=net0,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=hostnet0 95 96 -device megasas,id=scsi0,bus=pcie.0,addr=0x0 97 -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none 98 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 99 100Here is a full example with two different storage controllers on 101different PHBs, each with a disk, the second PHB is empty : 102 103.. code-block:: bash 104 105 $ qemu-system-ppc64 -m 2G -machine powernv9 -smp 2,cores=2,threads=1 -accel tcg,thread=single \ 106 -kernel ./zImage.epapr -initrd ./rootfs.cpio.xz -bios ./skiboot.lid \ 107 \ 108 -device megasas,id=scsi0,bus=pcie.0,addr=0x0 \ 109 -drive file=./rhel7-ppc64le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none \ 110 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 \ 111 \ 112 -device pcie-pci-bridge,id=bridge1,bus=pcie.1,addr=0x0 \ 113 \ 114 -device ich9-ahci,id=sata0,bus=bridge1,addr=0x1 \ 115 -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive0,format=qcow2,cache=none \ 116 -device ide-hd,bus=sata0.0,unit=0,drive=drive0,id=ide,bootindex=1 \ 117 -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=bridge1,addr=0x2 \ 118 -netdev bridge,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=net0 \ 119 -device nec-usb-xhci,bus=bridge1,addr=0x7 \ 120 \ 121 -serial mon:stdio -nographic 122 123You can also use VIRTIO devices : 124 125.. code-block:: bash 126 127 -drive file=./fedora-ppc64le.qcow2,if=none,snapshot=on,id=drive0 \ 128 -device virtio-blk-pci,drive=drive0,id=blk0,bus=pcie.0 \ 129 \ 130 -netdev tap,helper=/usr/lib/qemu/qemu-bridge-helper,br=virbr0,id=netdev0 \ 131 -device virtio-net-pci,netdev=netdev0,id=net0,bus=pcie.1 \ 132 \ 133 -fsdev local,id=fsdev0,path=$HOME,security_model=passthrough \ 134 -device virtio-9p-pci,fsdev=fsdev0,mount_tag=host,bus=pcie.2 135 136Multi sockets 137~~~~~~~~~~~~~ 138 139The number of sockets is deduced from the number of CPUs and the 140number of cores. ``-smp 2,cores=1`` will define a machine with 2 141sockets of 1 core, whereas ``-smp 2,cores=2`` will define a machine 142with 1 socket of 2 cores. ``-smp 8,cores=2``, 4 sockets of 2 cores. 143 144BMC configuration 145~~~~~~~~~~~~~~~~~ 146 147OpenPOWER systems negotiate the shutdown and reboot with their 148BMC. The QEMU PowerNV machine embeds an IPMI BMC simulator using the 149iBT interface and should offer the same power features. 150 151If you want to define your own BMC, use ``-nodefaults`` and specify 152one on the command line : 153 154.. code-block:: bash 155 156 -device ipmi-bmc-sim,id=bmc0 -device isa-ipmi-bt,bmc=bmc0,irq=10 157 158The files `palmetto-SDR.bin <http://www.kaod.org/qemu/powernv/palmetto-SDR.bin>`__ 159and `palmetto-FRU.bin <http://www.kaod.org/qemu/powernv/palmetto-FRU.bin>`__ 160define a Sensor Data Record repository and a Field Replaceable Unit 161inventory for a palmetto BMC. They can be used to extend the QEMU BMC 162simulator. 163 164.. code-block:: bash 165 166 -device ipmi-bmc-sim,sdrfile=./palmetto-SDR.bin,fruareasize=256,frudatafile=./palmetto-FRU.bin,id=bmc0 \ 167 -device isa-ipmi-bt,bmc=bmc0,irq=10 168 169The PowerNV machine can also be run with an external IPMI BMC device 170connected to a remote QEMU machine acting as BMC, using these options 171: 172 173.. code-block:: bash 174 175 -chardev socket,id=ipmi0,host=localhost,port=9002,reconnect=10 \ 176 -device ipmi-bmc-extern,id=bmc0,chardev=ipmi0 \ 177 -device isa-ipmi-bt,bmc=bmc0,irq=10 \ 178 -nodefaults 179 180NVRAM 181~~~~~ 182 183Use a MTD drive to add a PNOR to the machine, and get a NVRAM : 184 185.. code-block:: bash 186 187 -drive file=./witherspoon.pnor,format=raw,if=mtd 188 189CAVEATS 190------- 191 192 * No support for multiple HW threads (SMT=1). Same as pseries. 193 * CPU can hang when doing intensive I/Os. Use ``-append powersave=off`` in that case. 194