1Recommendations for KVM CPU model configuration on x86 hosts 2============================================================ 3 4The information that follows provides recommendations for configuring 5CPU models on x86 hosts. The goals are to maximise performance, while 6protecting guest OS against various CPU hardware flaws, and optionally 7enabling live migration between hosts with heterogeneous CPU models. 8 9 10Two ways to configure CPU models with QEMU / KVM 11^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 12 13(1) **Host passthrough** 14 15 This passes the host CPU model features, model, stepping, exactly to 16 the guest. Note that KVM may filter out some host CPU model features 17 if they cannot be supported with virtualization. Live migration is 18 unsafe when this mode is used as libvirt / QEMU cannot guarantee a 19 stable CPU is exposed to the guest across hosts. This is the 20 recommended CPU to use, provided live migration is not required. 21 22(2) **Named model** 23 24 QEMU comes with a number of predefined named CPU models, that 25 typically refer to specific generations of hardware released by 26 Intel and AMD. These allow the guest VMs to have a degree of 27 isolation from the host CPU, allowing greater flexibility in live 28 migrating between hosts with differing hardware. @end table 29 30In both cases, it is possible to optionally add or remove individual CPU 31features, to alter what is presented to the guest by default. 32 33Libvirt supports a third way to configure CPU models known as "Host 34model". This uses the QEMU "Named model" feature, automatically picking 35a CPU model that is similar the host CPU, and then adding extra features 36to approximate the host model as closely as possible. This does not 37guarantee the CPU family, stepping, etc will precisely match the host 38CPU, as they would with "Host passthrough", but gives much of the 39benefit of passthrough, while making live migration safe. 40 41 42ABI compatibility levels for CPU models 43^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 44 45The x86_64 architecture has a number of `ABI compatibility levels`_ 46defined. Traditionally most operating systems and toolchains would 47only target the original baseline ABI. It is expected that in 48future OS and toolchains are likely to target newer ABIs. The 49table that follows illustrates which ABI compatibility levels 50can be satisfied by the QEMU CPU models. Note that the table only 51lists the long term stable CPU model versions (eg Haswell-v4). 52In addition to whats listed, there are also many CPU model 53aliases which resolve to a different CPU model version, 54depending on the machine type is in use. 55 56.. _ABI compatibility levels: https://gitlab.com/x86-psABIs/x86-64-ABI/ 57 58.. csv-table:: x86-64 ABI compatibility levels 59 :file: cpu-models-x86-abi.csv 60 :widths: 40,15,15,15,15 61 :header-rows: 2 62 63 64Preferred CPU models for Intel x86 hosts 65^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 66 67The following CPU models are preferred for use on Intel hosts. 68Administrators / applications are recommended to use the CPU model that 69matches the generation of the host CPUs in use. In a deployment with a 70mixture of host CPU models between machines, if live migration 71compatibility is required, use the newest CPU model that is compatible 72across all desired hosts. 73 74``Cascadelake-Server``, ``Cascadelake-Server-noTSX`` 75 Intel Xeon Processor (Cascade Lake, 2019), with "stepping" levels 6 76 or 7 only. (The Cascade Lake Xeon processor with *stepping 5 is 77 vulnerable to MDS variants*.) 78 79``Skylake-Server``, ``Skylake-Server-IBRS``, ``Skylake-Server-IBRS-noTSX`` 80 Intel Xeon Processor (Skylake, 2016) 81 82``Skylake-Client``, ``Skylake-Client-IBRS``, ``Skylake-Client-noTSX-IBRS}`` 83 Intel Core Processor (Skylake, 2015) 84 85``Broadwell``, ``Broadwell-IBRS``, ``Broadwell-noTSX``, ``Broadwell-noTSX-IBRS`` 86 Intel Core Processor (Broadwell, 2014) 87 88``Haswell``, ``Haswell-IBRS``, ``Haswell-noTSX``, ``Haswell-noTSX-IBRS`` 89 Intel Core Processor (Haswell, 2013) 90 91``IvyBridge``, ``IvyBridge-IBR`` 92 Intel Xeon E3-12xx v2 (Ivy Bridge, 2012) 93 94``SandyBridge``, ``SandyBridge-IBRS`` 95 Intel Xeon E312xx (Sandy Bridge, 2011) 96 97``Westmere``, ``Westmere-IBRS`` 98 Westmere E56xx/L56xx/X56xx (Nehalem-C, 2010) 99 100``Nehalem``, ``Nehalem-IBRS`` 101 Intel Core i7 9xx (Nehalem Class Core i7, 2008) 102 103``Penryn`` 104 Intel Core 2 Duo P9xxx (Penryn Class Core 2, 2007) 105 106``Conroe`` 107 Intel Celeron_4x0 (Conroe/Merom Class Core 2, 2006) 108 109 110Important CPU features for Intel x86 hosts 111^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 112 113The following are important CPU features that should be used on Intel 114x86 hosts, when available in the host CPU. Some of them require explicit 115configuration to enable, as they are not included by default in some, or 116all, of the named CPU models listed above. In general all of these 117features are included if using "Host passthrough" or "Host model". 118 119``pcid`` 120 Recommended to mitigate the cost of the Meltdown (CVE-2017-5754) fix. 121 122 Included by default in Haswell, Broadwell & Skylake Intel CPU models. 123 124 Should be explicitly turned on for Westmere, SandyBridge, and 125 IvyBridge Intel CPU models. Note that some desktop/mobile Westmere 126 CPUs cannot support this feature. 127 128``spec-ctrl`` 129 Required to enable the Spectre v2 (CVE-2017-5715) fix. 130 131 Included by default in Intel CPU models with -IBRS suffix. 132 133 Must be explicitly turned on for Intel CPU models without -IBRS 134 suffix. 135 136 Requires the host CPU microcode to support this feature before it 137 can be used for guest CPUs. 138 139``stibp`` 140 Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some 141 operating systems. 142 143 Must be explicitly turned on for all Intel CPU models. 144 145 Requires the host CPU microcode to support this feature before it can 146 be used for guest CPUs. 147 148``ssbd`` 149 Required to enable the CVE-2018-3639 fix. 150 151 Not included by default in any Intel CPU model. 152 153 Must be explicitly turned on for all Intel CPU models. 154 155 Requires the host CPU microcode to support this feature before it 156 can be used for guest CPUs. 157 158``pdpe1gb`` 159 Recommended to allow guest OS to use 1GB size pages. 160 161 Not included by default in any Intel CPU model. 162 163 Should be explicitly turned on for all Intel CPU models. 164 165 Note that not all CPU hardware will support this feature. 166 167``md-clear`` 168 Required to confirm the MDS (CVE-2018-12126, CVE-2018-12127, 169 CVE-2018-12130, CVE-2019-11091) fixes. 170 171 Not included by default in any Intel CPU model. 172 173 Must be explicitly turned on for all Intel CPU models. 174 175 Requires the host CPU microcode to support this feature before it 176 can be used for guest CPUs. 177 178``mds-no`` 179 Recommended to inform the guest OS that the host is *not* vulnerable 180 to any of the MDS variants ([MFBDS] CVE-2018-12130, [MLPDS] 181 CVE-2018-12127, [MSBDS] CVE-2018-12126). 182 183 This is an MSR (Model-Specific Register) feature rather than a CPUID feature, 184 so it will not appear in the Linux ``/proc/cpuinfo`` in the host or 185 guest. Instead, the host kernel uses it to populate the MDS 186 vulnerability file in ``sysfs``. 187 188 So it should only be enabled for VMs if the host reports @code{Not 189 affected} in the ``/sys/devices/system/cpu/vulnerabilities/mds`` file. 190 191``taa-no`` 192 Recommended to inform that the guest that the host is ``not`` 193 vulnerable to CVE-2019-11135, TSX Asynchronous Abort (TAA). 194 195 This too is an MSR feature, so it does not show up in the Linux 196 ``/proc/cpuinfo`` in the host or guest. 197 198 It should only be enabled for VMs if the host reports ``Not affected`` 199 in the ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` 200 file. 201 202``tsx-ctrl`` 203 Recommended to inform the guest that it can disable the Intel TSX 204 (Transactional Synchronization Extensions) feature; or, if the 205 processor is vulnerable, use the Intel VERW instruction (a 206 processor-level instruction that performs checks on memory access) as 207 a mitigation for the TAA vulnerability. (For details, refer to 208 Intel's `deep dive into MDS 209 <https://software.intel.com/security-software-guidance/insights/deep-dive-intel-analysis-microarchitectural-data-sampling>`_.) 210 211 Expose this to the guest OS if and only if: (a) the host has TSX 212 enabled; *and* (b) the guest has ``rtm`` CPU flag enabled. 213 214 By disabling TSX, KVM-based guests can avoid paying the price of 215 mitigating TSX-based attacks. 216 217 Note that ``tsx-ctrl`` too is an MSR feature, so it does not show 218 up in the Linux ``/proc/cpuinfo`` in the host or guest. 219 220 To validate that Intel TSX is indeed disabled for the guest, there are 221 two ways: (a) check for the *absence* of ``rtm`` in the guest's 222 ``/proc/cpuinfo``; or (b) the 223 ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` file in 224 the guest should report ``Mitigation: TSX disabled``. 225 226 227Preferred CPU models for AMD x86 hosts 228^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 229 230The following CPU models are preferred for use on AMD hosts. 231Administrators / applications are recommended to use the CPU model that 232matches the generation of the host CPUs in use. In a deployment with a 233mixture of host CPU models between machines, if live migration 234compatibility is required, use the newest CPU model that is compatible 235across all desired hosts. 236 237``EPYC``, ``EPYC-IBPB`` 238 AMD EPYC Processor (2017) 239 240``Opteron_G5`` 241 AMD Opteron 63xx class CPU (2012) 242 243``Opteron_G4`` 244 AMD Opteron 62xx class CPU (2011) 245 246``Opteron_G3`` 247 AMD Opteron 23xx (Gen 3 Class Opteron, 2009) 248 249``Opteron_G2`` 250 AMD Opteron 22xx (Gen 2 Class Opteron, 2006) 251 252``Opteron_G1`` 253 AMD Opteron 240 (Gen 1 Class Opteron, 2004) 254 255 256Important CPU features for AMD x86 hosts 257^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 258 259The following are important CPU features that should be used on AMD x86 260hosts, when available in the host CPU. Some of them require explicit 261configuration to enable, as they are not included by default in some, or 262all, of the named CPU models listed above. In general all of these 263features are included if using "Host passthrough" or "Host model". 264 265``ibpb`` 266 Required to enable the Spectre v2 (CVE-2017-5715) fix. 267 268 Included by default in AMD CPU models with -IBPB suffix. 269 270 Must be explicitly turned on for AMD CPU models without -IBPB suffix. 271 272 Requires the host CPU microcode to support this feature before it 273 can be used for guest CPUs. 274 275``stibp`` 276 Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some 277 operating systems. 278 279 Must be explicitly turned on for all AMD CPU models. 280 281 Requires the host CPU microcode to support this feature before it 282 can be used for guest CPUs. 283 284``virt-ssbd`` 285 Required to enable the CVE-2018-3639 fix 286 287 Not included by default in any AMD CPU model. 288 289 Must be explicitly turned on for all AMD CPU models. 290 291 This should be provided to guests, even if amd-ssbd is also provided, 292 for maximum guest compatibility. 293 294 Note for some QEMU / libvirt versions, this must be force enabled when 295 when using "Host model", because this is a virtual feature that 296 doesn't exist in the physical host CPUs. 297 298``amd-ssbd`` 299 Required to enable the CVE-2018-3639 fix 300 301 Not included by default in any AMD CPU model. 302 303 Must be explicitly turned on for all AMD CPU models. 304 305 This provides higher performance than ``virt-ssbd`` so should be 306 exposed to guests whenever available in the host. ``virt-ssbd`` should 307 none the less also be exposed for maximum guest compatibility as some 308 kernels only know about ``virt-ssbd``. 309 310``amd-no-ssb`` 311 Recommended to indicate the host is not vulnerable CVE-2018-3639 312 313 Not included by default in any AMD CPU model. 314 315 Future hardware generations of CPU will not be vulnerable to 316 CVE-2018-3639, and thus the guest should be told not to enable 317 its mitigations, by exposing amd-no-ssb. This is mutually 318 exclusive with virt-ssbd and amd-ssbd. 319 320``pdpe1gb`` 321 Recommended to allow guest OS to use 1GB size pages 322 323 Not included by default in any AMD CPU model. 324 325 Should be explicitly turned on for all AMD CPU models. 326 327 Note that not all CPU hardware will support this feature. 328 329 330Default x86 CPU models 331^^^^^^^^^^^^^^^^^^^^^^ 332 333The default QEMU CPU models are designed such that they can run on all 334hosts. If an application does not wish to do perform any host 335compatibility checks before launching guests, the default is guaranteed 336to work. 337 338The default CPU models will, however, leave the guest OS vulnerable to 339various CPU hardware flaws, so their use is strongly discouraged. 340Applications should follow the earlier guidance to setup a better CPU 341configuration, with host passthrough recommended if live migration is 342not needed. 343 344``qemu32``, ``qemu64`` 345 QEMU Virtual CPU version 2.5+ (32 & 64 bit variants) 346 347``qemu64`` is used for x86_64 guests and ``qemu32`` is used for i686 348guests, when no ``-cpu`` argument is given to QEMU, or no ``<cpu>`` is 349provided in libvirt XML. 350 351Other non-recommended x86 CPUs 352^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 353 354The following CPUs models are compatible with most AMD and Intel x86 355hosts, but their usage is discouraged, as they expose a very limited 356featureset, which prevents guests having optimal performance. 357 358``kvm32``, ``kvm64`` 359 Common KVM processor (32 & 64 bit variants). 360 361 Legacy models just for historical compatibility with ancient QEMU 362 versions. 363 364``486``, ``athlon``, ``phenom``, ``coreduo``, ``core2duo``, ``n270``, ``pentium``, ``pentium2``, ``pentium3`` 365 Various very old x86 CPU models, mostly predating the introduction 366 of hardware assisted virtualization, that should thus not be 367 required for running virtual machines. 368 369 370Syntax for configuring CPU models 371================================= 372 373The examples below illustrate the approach to configuring the various 374CPU models / features in QEMU and libvirt. 375 376QEMU command line 377^^^^^^^^^^^^^^^^^ 378 379Host passthrough: 380 381.. parsed-literal:: 382 383 |qemu_system| -cpu host 384 385Host passthrough with feature customization: 386 387.. parsed-literal:: 388 389 |qemu_system| -cpu host,vmx=off,... 390 391Named CPU models: 392 393.. parsed-literal:: 394 395 |qemu_system| -cpu Westmere 396 397Named CPU models with feature customization: 398 399.. parsed-literal:: 400 401 |qemu_system| -cpu Westmere,pcid=on,... 402 403Libvirt guest XML 404^^^^^^^^^^^^^^^^^ 405 406Host passthrough:: 407 408 <cpu mode='host-passthrough'/> 409 410Host passthrough with feature customization:: 411 412 <cpu mode='host-passthrough'> 413 <feature name="vmx" policy="disable"/> 414 ... 415 </cpu> 416 417Host model:: 418 419 <cpu mode='host-model'/> 420 421Host model with feature customization:: 422 423 <cpu mode='host-model'> 424 <feature name="vmx" policy="disable"/> 425 ... 426 </cpu> 427 428Named model:: 429 430 <cpu mode='custom'> 431 <model name="Westmere"/> 432 </cpu> 433 434Named model with feature customization:: 435 436 <cpu mode='custom'> 437 <model name="Westmere"/> 438 <feature name="pcid" policy="require"/> 439 ... 440 </cpu> 441