1 .. _cpu-topology-s390x: 2 3 CPU topology on s390x 4 ===================== 5 6 Since QEMU 8.2, CPU topology on s390x provides up to 3 levels of 7 topology containers: drawers, books and sockets. They define a 8 tree-shaped hierarchy. 9 10 The socket container has one or more CPU entries. 11 Each of these CPU entries consists of a bitmap and three CPU attributes: 12 13 - CPU type 14 - entitlement 15 - dedication 16 17 Each bit set in the bitmap correspond to a core-id of a vCPU with matching 18 attributes. 19 20 This documentation provides general information on S390 CPU topology, 21 how to enable it and explains the new CPU attributes. 22 For information on how to modify the S390 CPU topology and how to 23 monitor polarization changes, see ``docs/devel/s390-cpu-topology.rst``. 24 25 Prerequisites 26 ------------- 27 28 To use the CPU topology, you need to run with KVM on a s390x host that 29 uses the Linux kernel v6.0 or newer (which provide the so-called 30 ``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the 31 CPU topology facility via the so-called STFLE bit 11 to the VM). 32 33 Enabling CPU topology 34 --------------------- 35 36 Currently, CPU topology is only enabled in the host model by default. 37 38 Enabling CPU topology in a CPU model is done by setting the CPU flag 39 ``ctop`` to ``on`` as in: 40 41 .. code-block:: bash 42 43 -cpu gen16b,ctop=on 44 45 Having the topology disabled by default allows migration between 46 old and new QEMU without adding new flags. 47 48 Default topology usage 49 ---------------------- 50 51 The CPU topology can be specified on the QEMU command line 52 with the ``-smp`` or the ``-device`` QEMU command arguments. 53 54 Note also that since 7.2 threads are no longer supported in the topology 55 and the ``-smp`` command line argument accepts only ``threads=1``. 56 57 If none of the containers attributes (drawers, books, sockets) are 58 specified for the ``-smp`` flag, the number of these containers 59 is 1. 60 61 Thus the following two options will result in the same topology: 62 63 .. code-block:: bash 64 65 -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32 66 67 and 68 69 .. code-block:: bash 70 71 -smp cpus=5,sockets=8,cores=4,maxcpus=32 72 73 When a CPU is defined by the ``-smp`` command argument, its position 74 inside the topology is calculated by adding the CPUs to the topology 75 based on the core-id starting with core-0 at position 0 of socket-0, 76 book-0, drawer-0 and filling all CPUs of socket-0 before filling socket-1 77 of book-0 and so on up to the last socket of the last book of the last 78 drawer. 79 80 When a CPU is defined by the ``-device`` command argument, the 81 tree topology attributes must all be defined or all not defined. 82 83 .. code-block:: bash 84 85 -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 86 87 or 88 89 .. code-block:: bash 90 91 -device gen16b-s390x-cpu,core-id=1,dedicated=true 92 93 If none of the tree attributes (drawer, book, sockets), are specified 94 for the ``-device`` argument, like for all CPUs defined with the ``-smp`` 95 command argument the topology tree attributes will be set by simply 96 adding the CPUs to the topology based on the core-id. 97 98 QEMU will not try to resolve collisions and will report an error if the 99 CPU topology defined explicitly or implicitly on a ``-device`` 100 argument collides with the definition of a CPU implicitly defined 101 on the ``-smp`` argument. 102 103 When the topology modifier attributes are not defined for the 104 ``-device`` command argument they takes following default values: 105 106 - dedicated: ``false`` 107 - entitlement: ``medium`` 108 109 110 Hot plug 111 ++++++++ 112 113 New CPUs can be plugged using the device_add hmp command as in: 114 115 .. code-block:: bash 116 117 (qemu) device_add gen16b-s390x-cpu,core-id=9 118 119 The placement of the CPU is derived from the core-id as described above. 120 121 The topology can of course also be fully defined: 122 123 .. code-block:: bash 124 125 (qemu) device_add gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 126 127 128 Examples 129 ++++++++ 130 131 In the following machine we define 8 sockets with 4 cores each. 132 133 .. code-block:: bash 134 135 $ qemu-system-s390x -m 2G \ 136 -cpu gen16b,ctop=on \ 137 -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ 138 -device host-s390x-cpu,core-id=14 \ 139 140 A new CPUs can be plugged using the device_add hmp command as before: 141 142 .. code-block:: bash 143 144 (qemu) device_add gen16b-s390x-cpu,core-id=9 145 146 The core-id defines the placement of the core in the topology by 147 starting with core 0 in socket 0 up to maxcpus. 148 149 In the example above: 150 151 * There are 5 CPUs provided to the guest with the ``-smp`` command line 152 They will take the core-ids 0,1,2,3,4 153 As we have 4 cores in a socket, we have 4 CPUs provided 154 to the guest in socket 0, with core-ids 0,1,2,3. 155 The last CPU, with core-id 4, will be on socket 1. 156 157 * the core with ID 14 provided by the ``-device`` command line will 158 be placed in socket 3, with core-id 14 159 160 * the core with ID 9 provided by the ``device_add`` qmp command will 161 be placed in socket 2, with core-id 9 162 163 164 Polarization, entitlement and dedication 165 ---------------------------------------- 166 167 Polarization 168 ++++++++++++ 169 170 The polarization affects how the CPUs of a shared host are utilized/distributed 171 among guests. 172 The guest determines the polarization by using the PTF instruction. 173 174 Polarization defines two models of CPU provisioning: horizontal 175 and vertical. 176 177 The horizontal polarization is the default model on boot and after 178 subsystem reset. When horizontal polarization is in effect all vCPUs should 179 have about equal resource provisioning. 180 181 In the vertical polarization model vCPUs are unequal, but overall more resources 182 might be available. 183 The guest can make use of the vCPU entitlement information provided by the host 184 to optimize kernel thread scheduling. 185 186 A subsystem reset puts all vCPU of the configuration into the 187 horizontal polarization. 188 189 Entitlement 190 +++++++++++ 191 192 The vertical polarization specifies that the guest's vCPU can get 193 different real CPU provisioning: 194 195 - a vCPU with vertical high entitlement specifies that this 196 vCPU gets 100% of the real CPU provisioning. 197 198 - a vCPU with vertical medium entitlement specifies that this 199 vCPU shares the real CPU with other vCPUs. 200 201 - a vCPU with vertical low entitlement specifies that this 202 vCPU only gets real CPU provisioning when no other vCPUs needs it. 203 204 In the case a vCPU with vertical high entitlement does not use 205 the real CPU, the unused "slack" can be dispatched to other vCPU 206 with medium or low entitlement. 207 208 A vCPU can be "dedicated" in which case the vCPU is fully dedicated to a single 209 real CPU. 210 211 The dedicated bit is an indication of affinity of a vCPU for a real CPU 212 while the entitlement indicates the sharing or exclusivity of use. 213 214 Defining the topology on the command line 215 ----------------------------------------- 216 217 The topology can entirely be defined using -device cpu statements, 218 with the exception of CPU 0 which must be defined with the -smp 219 argument. 220 221 For example, here we set the position of the cores 1,2,3 to 222 drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0, 223 book 0, socket 0 without defining entitlement or dedication. 224 Core 4 will be set on its default position on socket 1 225 (since we have 4 core per socket) and we define it as dedicated and 226 with vertical high entitlement. 227 228 .. code-block:: bash 229 230 $ qemu-system-s390x -m 2G \ 231 -cpu gen16b,ctop=on \ 232 -smp cpus=1,sockets=8,cores=4,maxcpus=32 \ 233 \ 234 -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \ 235 -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \ 236 -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \ 237 \ 238 -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \ 239 -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \ 240 \ 241 -device gen16b-s390x-cpu,core-id=4,dedicated=on,entitlement=high 242 243 The entitlement defined for the CPU 4 will only be used after the guest 244 successfully enables vertical polarization by using the PTF instruction. 245