1 PowerNV family boards (``powernv8``, ``powernv9``) 2 ================================================================== 3 4 PowerNV (as Non-Virtualized) is the "baremetal" platform using the 5 OPAL firmware. It runs Linux on IBM and OpenPOWER systems and it can 6 be used as an hypervisor OS, running KVM guests, or simply as a host 7 OS. 8 9 The PowerNV QEMU machine tries to emulate a PowerNV system at the 10 level of the skiboot firmware, which loads the OS and provides some 11 runtime services. Power Systems have a lower firmware (HostBoot) that 12 does low level system initialization, like DRAM training. This is 13 beyond the scope of what QEMU addresses today. 14 15 Supported devices 16 ----------------- 17 18 * Multi processor support for POWER8, POWER8NVL and POWER9. 19 * XSCOM, serial communication sideband bus to configure chiplets 20 * Simple LPC Controller 21 * Processor Service Interface (PSI) Controller 22 * Interrupt Controller, XICS (POWER8) and XIVE (POWER9) 23 * POWER8 PHB3 PCIe Host bridge and POWER9 PHB4 PCIe Host bridge 24 * Simple OCC is an on-chip microcontroller used for power management 25 tasks 26 * iBT device to handle BMC communication, with the internal BMC 27 simulator provided by QEMU or an external BMC such as an Aspeed 28 QEMU machine. 29 * PNOR containing the different firmware partitions. 30 31 Missing devices 32 --------------- 33 34 A lot is missing, among which : 35 36 * POWER10 processor 37 * XIVE2 (POWER10) interrupt controller 38 * I2C controllers (yet to be merged) 39 * NPU/NPU2/NPU3 controllers 40 * EEH support for PCIe Host bridge controllers 41 * NX controller 42 * VAS controller 43 * chipTOD (Time Of Day) 44 * Self Boot Engine (SBE). 45 * FSI bus 46 47 Firmware 48 -------- 49 50 The OPAL firmware (OpenPower Abstraction Layer) for OpenPower systems 51 includes the runtime services ``skiboot`` and the bootloader kernel and 52 initramfs ``skiroot``. Source code can be found on GitHub: 53 54 https://github.com/open-power. 55 56 Prebuilt images of ``skiboot`` and ``skiroot`` are made available on the `OpenPOWER <https://github.com/open-power/op-build/releases/>`__ site. 57 58 QEMU includes a prebuilt image of ``skiboot`` which is updated when a 59 more recent version is required by the models. 60 61 Boot options 62 ------------ 63 64 Here is a simple setup with one e1000e NIC : 65 66 .. code-block:: bash 67 68 $ qemu-system-ppc64 -m 2G -machine powernv9 -smp 2,cores=2,threads=1 \ 69 -accel tcg,thread=single \ 70 -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 \ 71 -netdev user,id=net0,hostfwd=::20022-:22,hostname=pnv \ 72 -kernel ./zImage.epapr \ 73 -initrd ./rootfs.cpio.xz \ 74 -nographic 75 76 and a SATA disk : 77 78 .. code-block:: bash 79 80 -device ich9-ahci,id=sata0,bus=pcie.1,addr=0x0 \ 81 -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive0,format=qcow2,cache=none \ 82 -device ide-hd,bus=sata0.0,unit=0,drive=drive0,id=ide,bootindex=1 \ 83 84 Complex PCIe configuration 85 ~~~~~~~~~~~~~~~~~~~~~~~~~~ 86 Six PHBs are defined per chip (POWER9) but no default PCI layout is 87 provided (to be compatible with libvirt). One PCI device can be added 88 on any of the available PCIe slots using command line options such as: 89 90 .. code-block:: bash 91 92 -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 93 -netdev bridge,id=net0,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=hostnet0 94 95 -device megasas,id=scsi0,bus=pcie.0,addr=0x0 96 -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none 97 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 98 99 Here is a full example with two different storage controllers on 100 different PHBs, each with a disk, the second PHB is empty : 101 102 .. code-block:: bash 103 104 $ qemu-system-ppc64 -m 2G -machine powernv9 -smp 2,cores=2,threads=1 -accel tcg,thread=single \ 105 -kernel ./zImage.epapr -initrd ./rootfs.cpio.xz -bios ./skiboot.lid \ 106 \ 107 -device megasas,id=scsi0,bus=pcie.0,addr=0x0 \ 108 -drive file=./rhel7-ppc64le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none \ 109 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 \ 110 \ 111 -device pcie-pci-bridge,id=bridge1,bus=pcie.1,addr=0x0 \ 112 \ 113 -device ich9-ahci,id=sata0,bus=bridge1,addr=0x1 \ 114 -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive0,format=qcow2,cache=none \ 115 -device ide-hd,bus=sata0.0,unit=0,drive=drive0,id=ide,bootindex=1 \ 116 -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=bridge1,addr=0x2 \ 117 -netdev bridge,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=net0 \ 118 -device nec-usb-xhci,bus=bridge1,addr=0x7 \ 119 \ 120 -serial mon:stdio -nographic 121 122 You can also use VIRTIO devices : 123 124 .. code-block:: bash 125 126 -drive file=./fedora-ppc64le.qcow2,if=none,snapshot=on,id=drive0 \ 127 -device virtio-blk-pci,drive=drive0,id=blk0,bus=pcie.0 \ 128 \ 129 -netdev tap,helper=/usr/lib/qemu/qemu-bridge-helper,br=virbr0,id=netdev0 \ 130 -device virtio-net-pci,netdev=netdev0,id=net0,bus=pcie.1 \ 131 \ 132 -fsdev local,id=fsdev0,path=$HOME,security_model=passthrough \ 133 -device virtio-9p-pci,fsdev=fsdev0,mount_tag=host,bus=pcie.2 134 135 Multi sockets 136 ~~~~~~~~~~~~~ 137 138 The number of sockets is deduced from the number of CPUs and the 139 number of cores. ``-smp 2,cores=1`` will define a machine with 2 140 sockets of 1 core, whereas ``-smp 2,cores=2`` will define a machine 141 with 1 socket of 2 cores. ``-smp 8,cores=2``, 4 sockets of 2 cores. 142 143 BMC configuration 144 ~~~~~~~~~~~~~~~~~ 145 146 OpenPOWER systems negotiate the shutdown and reboot with their 147 BMC. The QEMU PowerNV machine embeds an IPMI BMC simulator using the 148 iBT interface and should offer the same power features. 149 150 If you want to define your own BMC, use ``-nodefaults`` and specify 151 one on the command line : 152 153 .. code-block:: bash 154 155 -device ipmi-bmc-sim,id=bmc0 -device isa-ipmi-bt,bmc=bmc0,irq=10 156 157 The files `palmetto-SDR.bin <http://www.kaod.org/qemu/powernv/palmetto-SDR.bin>`__ 158 and `palmetto-FRU.bin <http://www.kaod.org/qemu/powernv/palmetto-FRU.bin>`__ 159 define a Sensor Data Record repository and a Field Replaceable Unit 160 inventory for a palmetto BMC. They can be used to extend the QEMU BMC 161 simulator. 162 163 .. code-block:: bash 164 165 -device ipmi-bmc-sim,sdrfile=./palmetto-SDR.bin,fruareasize=256,frudatafile=./palmetto-FRU.bin,id=bmc0 \ 166 -device isa-ipmi-bt,bmc=bmc0,irq=10 167 168 The PowerNV machine can also be run with an external IPMI BMC device 169 connected to a remote QEMU machine acting as BMC, using these options 170 : 171 172 .. code-block:: bash 173 174 -chardev socket,id=ipmi0,host=localhost,port=9002,reconnect=10 \ 175 -device ipmi-bmc-extern,id=bmc0,chardev=ipmi0 \ 176 -device isa-ipmi-bt,bmc=bmc0,irq=10 \ 177 -nodefaults 178 179 NVRAM 180 ~~~~~ 181 182 Use a MTD drive to add a PNOR to the machine, and get a NVRAM : 183 184 .. code-block:: bash 185 186 -drive file=./witherspoon.pnor,format=raw,if=mtd 187 188 CAVEATS 189 ------- 190 191 * No support for multiple HW threads (SMT=1). Same as pseries. 192 * CPU can hang when doing intensive I/Os. Use ``-append powersave=off`` in that case. 193