1Recommendations for KVM CPU model configuration on x86 hosts 2~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 3 4The information that follows provides recommendations for configuring 5CPU models on x86 hosts. The goals are to maximise performance, while 6protecting guest OS against various CPU hardware flaws, and optionally 7enabling live migration between hosts with heterogeneous CPU models. 8 9 10Two ways to configure CPU models with QEMU / KVM 11^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 12 13(1) **Host passthrough** 14 15 This passes the host CPU model features, model, stepping, exactly to 16 the guest. Note that KVM may filter out some host CPU model features 17 if they cannot be supported with virtualization. Live migration is 18 unsafe when this mode is used as libvirt / QEMU cannot guarantee a 19 stable CPU is exposed to the guest across hosts. This is the 20 recommended CPU to use, provided live migration is not required. 21 22(2) **Named model** 23 24 QEMU comes with a number of predefined named CPU models, that 25 typically refer to specific generations of hardware released by 26 Intel and AMD. These allow the guest VMs to have a degree of 27 isolation from the host CPU, allowing greater flexibility in live 28 migrating between hosts with differing hardware. @end table 29 30In both cases, it is possible to optionally add or remove individual CPU 31features, to alter what is presented to the guest by default. 32 33Libvirt supports a third way to configure CPU models known as "Host 34model". This uses the QEMU "Named model" feature, automatically picking 35a CPU model that is similar the host CPU, and then adding extra features 36to approximate the host model as closely as possible. This does not 37guarantee the CPU family, stepping, etc will precisely match the host 38CPU, as they would with "Host passthrough", but gives much of the 39benefit of passthrough, while making live migration safe. 40 41 42Preferred CPU models for Intel x86 hosts 43^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 44 45The following CPU models are preferred for use on Intel hosts. 46Administrators / applications are recommended to use the CPU model that 47matches the generation of the host CPUs in use. In a deployment with a 48mixture of host CPU models between machines, if live migration 49compatibility is required, use the newest CPU model that is compatible 50across all desired hosts. 51 52``Cascadelake-Server``, ``Cascadelake-Server-noTSX`` 53 Intel Xeon Processor (Cascade Lake, 2019), with "stepping" levels 6 54 or 7 only. (The Cascade Lake Xeon processor with *stepping 5 is 55 vulnerable to MDS variants*.) 56 57``Skylake-Server``, ``Skylake-Server-IBRS``, ``Skylake-Server-IBRS-noTSX`` 58 Intel Xeon Processor (Skylake, 2016) 59 60``Skylake-Client``, ``Skylake-Client-IBRS``, ``Skylake-Client-noTSX-IBRS}`` 61 Intel Core Processor (Skylake, 2015) 62 63``Broadwell``, ``Broadwell-IBRS``, ``Broadwell-noTSX``, ``Broadwell-noTSX-IBRS`` 64 Intel Core Processor (Broadwell, 2014) 65 66``Haswell``, ``Haswell-IBRS``, ``Haswell-noTSX``, ``Haswell-noTSX-IBRS`` 67 Intel Core Processor (Haswell, 2013) 68 69``IvyBridge``, ``IvyBridge-IBR`` 70 Intel Xeon E3-12xx v2 (Ivy Bridge, 2012) 71 72``SandyBridge``, ``SandyBridge-IBRS`` 73 Intel Xeon E312xx (Sandy Bridge, 2011) 74 75``Westmere``, ``Westmere-IBRS`` 76 Westmere E56xx/L56xx/X56xx (Nehalem-C, 2010) 77 78``Nehalem``, ``Nehalem-IBRS`` 79 Intel Core i7 9xx (Nehalem Class Core i7, 2008) 80 81``Penryn`` 82 Intel Core 2 Duo P9xxx (Penryn Class Core 2, 2007) 83 84``Conroe`` 85 Intel Celeron_4x0 (Conroe/Merom Class Core 2, 2006) 86 87 88Important CPU features for Intel x86 hosts 89^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 90 91The following are important CPU features that should be used on Intel 92x86 hosts, when available in the host CPU. Some of them require explicit 93configuration to enable, as they are not included by default in some, or 94all, of the named CPU models listed above. In general all of these 95features are included if using "Host passthrough" or "Host model". 96 97``pcid`` 98 Recommended to mitigate the cost of the Meltdown (CVE-2017-5754) fix. 99 100 Included by default in Haswell, Broadwell & Skylake Intel CPU models. 101 102 Should be explicitly turned on for Westmere, SandyBridge, and 103 IvyBridge Intel CPU models. Note that some desktop/mobile Westmere 104 CPUs cannot support this feature. 105 106``spec-ctrl`` 107 Required to enable the Spectre v2 (CVE-2017-5715) fix. 108 109 Included by default in Intel CPU models with -IBRS suffix. 110 111 Must be explicitly turned on for Intel CPU models without -IBRS 112 suffix. 113 114 Requires the host CPU microcode to support this feature before it 115 can be used for guest CPUs. 116 117``stibp`` 118 Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some 119 operating systems. 120 121 Must be explicitly turned on for all Intel CPU models. 122 123 Requires the host CPU microcode to support this feature before it can 124 be used for guest CPUs. 125 126``ssbd`` 127 Required to enable the CVE-2018-3639 fix. 128 129 Not included by default in any Intel CPU model. 130 131 Must be explicitly turned on for all Intel CPU models. 132 133 Requires the host CPU microcode to support this feature before it 134 can be used for guest CPUs. 135 136``pdpe1gb`` 137 Recommended to allow guest OS to use 1GB size pages. 138 139 Not included by default in any Intel CPU model. 140 141 Should be explicitly turned on for all Intel CPU models. 142 143 Note that not all CPU hardware will support this feature. 144 145``md-clear`` 146 Required to confirm the MDS (CVE-2018-12126, CVE-2018-12127, 147 CVE-2018-12130, CVE-2019-11091) fixes. 148 149 Not included by default in any Intel CPU model. 150 151 Must be explicitly turned on for all Intel CPU models. 152 153 Requires the host CPU microcode to support this feature before it 154 can be used for guest CPUs. 155 156``mds-no`` 157 Recommended to inform the guest OS that the host is *not* vulnerable 158 to any of the MDS variants ([MFBDS] CVE-2018-12130, [MLPDS] 159 CVE-2018-12127, [MSBDS] CVE-2018-12126). 160 161 This is an MSR (Model-Specific Register) feature rather than a CPUID feature, 162 so it will not appear in the Linux ``/proc/cpuinfo`` in the host or 163 guest. Instead, the host kernel uses it to populate the MDS 164 vulnerability file in ``sysfs``. 165 166 So it should only be enabled for VMs if the host reports @code{Not 167 affected} in the ``/sys/devices/system/cpu/vulnerabilities/mds`` file. 168 169``taa-no`` 170 Recommended to inform that the guest that the host is ``not`` 171 vulnerable to CVE-2019-11135, TSX Asynchronous Abort (TAA). 172 173 This too is an MSR feature, so it does not show up in the Linux 174 ``/proc/cpuinfo`` in the host or guest. 175 176 It should only be enabled for VMs if the host reports ``Not affected`` 177 in the ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` 178 file. 179 180``tsx-ctrl`` 181 Recommended to inform the guest that it can disable the Intel TSX 182 (Transactional Synchronization Extensions) feature; or, if the 183 processor is vulnerable, use the Intel VERW instruction (a 184 processor-level instruction that performs checks on memory access) as 185 a mitigation for the TAA vulnerability. (For details, refer to 186 Intel's `deep dive into MDS 187 <https://software.intel.com/security-software-guidance/insights/deep-dive-intel-analysis-microarchitectural-data-sampling>`_.) 188 189 Expose this to the guest OS if and only if: (a) the host has TSX 190 enabled; *and* (b) the guest has ``rtm`` CPU flag enabled. 191 192 By disabling TSX, KVM-based guests can avoid paying the price of 193 mitigating TSX-based attacks. 194 195 Note that ``tsx-ctrl`` too is an MSR feature, so it does not show 196 up in the Linux ``/proc/cpuinfo`` in the host or guest. 197 198 To validate that Intel TSX is indeed disabled for the guest, there are 199 two ways: (a) check for the *absence* of ``rtm`` in the guest's 200 ``/proc/cpuinfo``; or (b) the 201 ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` file in 202 the guest should report ``Mitigation: TSX disabled``. 203 204 205Preferred CPU models for AMD x86 hosts 206^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 207 208The following CPU models are preferred for use on Intel hosts. 209Administrators / applications are recommended to use the CPU model that 210matches the generation of the host CPUs in use. In a deployment with a 211mixture of host CPU models between machines, if live migration 212compatibility is required, use the newest CPU model that is compatible 213across all desired hosts. 214 215``EPYC``, ``EPYC-IBPB`` 216 AMD EPYC Processor (2017) 217 218``Opteron_G5`` 219 AMD Opteron 63xx class CPU (2012) 220 221``Opteron_G4`` 222 AMD Opteron 62xx class CPU (2011) 223 224``Opteron_G3`` 225 AMD Opteron 23xx (Gen 3 Class Opteron, 2009) 226 227``Opteron_G2`` 228 AMD Opteron 22xx (Gen 2 Class Opteron, 2006) 229 230``Opteron_G1`` 231 AMD Opteron 240 (Gen 1 Class Opteron, 2004) 232 233 234Important CPU features for AMD x86 hosts 235^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 236 237The following are important CPU features that should be used on AMD x86 238hosts, when available in the host CPU. Some of them require explicit 239configuration to enable, as they are not included by default in some, or 240all, of the named CPU models listed above. In general all of these 241features are included if using "Host passthrough" or "Host model". 242 243``ibpb`` 244 Required to enable the Spectre v2 (CVE-2017-5715) fix. 245 246 Included by default in AMD CPU models with -IBPB suffix. 247 248 Must be explicitly turned on for AMD CPU models without -IBPB suffix. 249 250 Requires the host CPU microcode to support this feature before it 251 can be used for guest CPUs. 252 253``stibp`` 254 Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some 255 operating systems. 256 257 Must be explicitly turned on for all AMD CPU models. 258 259 Requires the host CPU microcode to support this feature before it 260 can be used for guest CPUs. 261 262``virt-ssbd`` 263 Required to enable the CVE-2018-3639 fix 264 265 Not included by default in any AMD CPU model. 266 267 Must be explicitly turned on for all AMD CPU models. 268 269 This should be provided to guests, even if amd-ssbd is also provided, 270 for maximum guest compatibility. 271 272 Note for some QEMU / libvirt versions, this must be force enabled when 273 when using "Host model", because this is a virtual feature that 274 doesn't exist in the physical host CPUs. 275 276``amd-ssbd`` 277 Required to enable the CVE-2018-3639 fix 278 279 Not included by default in any AMD CPU model. 280 281 Must be explicitly turned on for all AMD CPU models. 282 283 This provides higher performance than ``virt-ssbd`` so should be 284 exposed to guests whenever available in the host. ``virt-ssbd`` should 285 none the less also be exposed for maximum guest compatibility as some 286 kernels only know about ``virt-ssbd``. 287 288``amd-no-ssb`` 289 Recommended to indicate the host is not vulnerable CVE-2018-3639 290 291 Not included by default in any AMD CPU model. 292 293 Future hardware generations of CPU will not be vulnerable to 294 CVE-2018-3639, and thus the guest should be told not to enable 295 its mitigations, by exposing amd-no-ssb. This is mutually 296 exclusive with virt-ssbd and amd-ssbd. 297 298``pdpe1gb`` 299 Recommended to allow guest OS to use 1GB size pages 300 301 Not included by default in any AMD CPU model. 302 303 Should be explicitly turned on for all AMD CPU models. 304 305 Note that not all CPU hardware will support this feature. 306 307 308Default x86 CPU models 309^^^^^^^^^^^^^^^^^^^^^^ 310 311The default QEMU CPU models are designed such that they can run on all 312hosts. If an application does not wish to do perform any host 313compatibility checks before launching guests, the default is guaranteed 314to work. 315 316The default CPU models will, however, leave the guest OS vulnerable to 317various CPU hardware flaws, so their use is strongly discouraged. 318Applications should follow the earlier guidance to setup a better CPU 319configuration, with host passthrough recommended if live migration is 320not needed. 321 322``qemu32``, ``qemu64`` 323 QEMU Virtual CPU version 2.5+ (32 & 64 bit variants) 324 325``qemu64`` is used for x86_64 guests and ``qemu32`` is used for i686 326guests, when no ``-cpu`` argument is given to QEMU, or no ``<cpu>`` is 327provided in libvirt XML. 328 329Other non-recommended x86 CPUs 330^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 331 332The following CPUs models are compatible with most AMD and Intel x86 333hosts, but their usage is discouraged, as they expose a very limited 334featureset, which prevents guests having optimal performance. 335 336``kvm32``, ``kvm64`` 337 Common KVM processor (32 & 64 bit variants). 338 339 Legacy models just for historical compatibility with ancient QEMU 340 versions. 341 342``486``, ``athlon``, ``phenom``, ``coreduo``, ``core2duo``, ``n270``, ``pentium``, ``pentium2``, ``pentium3`` 343 Various very old x86 CPU models, mostly predating the introduction 344 of hardware assisted virtualization, that should thus not be 345 required for running virtual machines. 346 347 348Syntax for configuring CPU models 349~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 350 351The examples below illustrate the approach to configuring the various 352CPU models / features in QEMU and libvirt. 353 354QEMU command line 355^^^^^^^^^^^^^^^^^ 356 357Host passthrough: 358 359.. parsed-literal:: 360 361 |qemu_system| -cpu host 362 363Host passthrough with feature customization: 364 365.. parsed-literal:: 366 367 |qemu_system| -cpu host,vmx=off,... 368 369Named CPU models: 370 371.. parsed-literal:: 372 373 |qemu_system| -cpu Westmere 374 375Named CPU models with feature customization: 376 377.. parsed-literal:: 378 379 |qemu_system| -cpu Westmere,pcid=on,... 380 381Libvirt guest XML 382^^^^^^^^^^^^^^^^^ 383 384Host passthrough:: 385 386 <cpu mode='host-passthrough'/> 387 388Host passthrough with feature customization:: 389 390 <cpu mode='host-passthrough'> 391 <feature name="vmx" policy="disable"/> 392 ... 393 </cpu> 394 395Host model:: 396 397 <cpu mode='host-model'/> 398 399Host model with feature customization:: 400 401 <cpu mode='host-model'> 402 <feature name="vmx" policy="disable"/> 403 ... 404 </cpu> 405 406Named model:: 407 408 <cpu mode='custom'> 409 <model name="Westmere"/> 410 </cpu> 411 412Named model with feature customization:: 413 414 <cpu mode='custom'> 415 <model name="Westmere"/> 416 <feature name="pcid" policy="require"/> 417 ... 418 </cpu> 419