Implement support for trapping REP CMPS instructions in NVMM.

Qemu would abort hard when NVMM would get a memory trap on the instruction
since it didn't know it.

Revert (REPE) CMPS support per request of Maxime, it is incorrect.

Implement missing (REPE) CMPS instruction support in NVMMs x86_decode().

In apparently rare cases the (REPE) CMPS instruction can trigger an memory
assist. NVMM wouldn't recognize the instruction an

Implement missing (REPE) CMPS instruction support in NVMMs x86_decode().

In apparently rare cases the (REPE) CMPS instruction can trigger an memory
assist. NVMM wouldn't recognize the instruction and thus couldn't assist and
Qemu would abort.

show more ...

nvmm: update copyright headers

Annotate a covering switch as such to avoid warnings about missing
returns.

Use the new PTE naming, and define CR3_FRAME_* separately. No functional
change.

Three changes in libnvmm:

- Add 'mach' and 'vcpu' backpointers in the nvmm_io and nvmm_mem
structures.

- Rename 'nvmm_callbacks' to 'nvmm_assist_callbacks'.

- Rename and migrate NVMM_MACH_CO

Three changes in libnvmm:

- Add 'mach' and 'vcpu' backpointers in the nvmm_io and nvmm_mem
structures.

- Rename 'nvmm_callbacks' to 'nvmm_assist_callbacks'.

- Rename and migrate NVMM_MACH_CONF_CALLBACKS to NVMM_VCPU_CONF_CALLBACKS,
it now becomes per-VCPU.

show more ...

Miscellaneous changes in NVMM, to address several inconsistencies and
issues in the libnvmm API.

- Rename NVMM_CAPABILITY_VERSION to NVMM_KERN_VERSION, and check it in
libnvmm. Introduce NVMM_US

Miscellaneous changes in NVMM, to address several inconsistencies and
issues in the libnvmm API.

- Rename NVMM_CAPABILITY_VERSION to NVMM_KERN_VERSION, and check it in
libnvmm. Introduce NVMM_USER_VERSION, for future use.

- In libnvmm, open "/dev/nvmm" as read-only and with O_CLOEXEC. This is to
avoid sharing the VMs with the children if the process forks. In the
NVMM driver, force O_CLOEXEC on open().

- Rename the following things for consistency:
nvmm_exit* -> nvmm_vcpu_exit*
nvmm_event* -> nvmm_vcpu_event*
NVMM_EXIT_* -> NVMM_VCPU_EXIT_*
NVMM_EVENT_INTERRUPT_HW -> NVMM_VCPU_EVENT_INTR
NVMM_EVENT_EXCEPTION -> NVMM_VCPU_EVENT_EXCP
Delete NVMM_EVENT_INTERRUPT_SW, unused already.

- Slightly reorganize the MI/MD definitions, for internal clarity.

- Split NVMM_VCPU_EXIT_MSR in two: NVMM_VCPU_EXIT_{RD,WR}MSR. Also provide
separate u.rdmsr and u.wrmsr fields. This is more consistent with the
other exit reasons.

- Change the types of several variables:
event.type enum -> u_int
event.vector uint64_t -> uint8_t
exit.u.*msr.msr: uint64_t -> uint32_t
exit.u.io.type: enum -> bool
exit.u.io.seg: int -> int8_t
cap.arch.mxcsr_mask: uint64_t -> uint32_t
cap.arch.conf_cpuid_maxops: uint64_t -> uint32_t

- Delete NVMM_VCPU_EXIT_MWAIT_COND, it is AMD-only and confusing, and we
already intercept 'monitor' so it is never armed.

- Introduce vmx_exit_insn() for NVMM-Intel, similar to svm_exit_insn().
The 'npc' field wasn't getting filled properly during certain VMEXITs.

- Introduce nvmm_vcpu_configure(). Similar to nvmm_machine_configure(),
but as its name indicates, the configuration is per-VCPU and not per-VM.
Migrate and rename NVMM_MACH_CONF_X86_CPUID to NVMM_VCPU_CONF_CPUID.
This becomes per-VCPU, which makes more sense than per-VM.

- Extend the NVMM_VCPU_CONF_CPUID conf to allow triggering VMEXITs on
specific leaves. Until now we could only mask the leaves. An uint32_t
is added in the structure:
uint32_t mask:1;
uint32_t exit:1;
uint32_t rsvd:30;
The two first bits select the desired behavior on the leaf. Specifying
zero on both resets the leaf to the default behavior. The new
NVMM_VCPU_EXIT_CPUID exit reason is added.

show more ...

Put back 'default', because llvm apparently doesn't realize that all cases
are covered in the switch.

Improve nvmm_vcpu_dump().

Implement XCHG, add associated tests, and add comments to explain. With
this in place the Windows 95 installer completes successfuly.

Part of PR/54611.

Fix incorrect parsing: the R/M field uses a special GPR map when the
address size is 16 bits, regardless of the actual operating mode. With
this special map there can be two registers referenced at o

Fix incorrect parsing: the R/M field uses a special GPR map when the
address size is 16 bits, regardless of the actual operating mode. With
this special map there can be two registers referenced at once, and
also disp16-only.

Implement this special behavior, and add associated tests. While here
simplify a few things.

With this in place, the Windows 95 installer initializes correctly.

Part of PR/54611.

show more ...

Change the NVMM API to reduce data movements. Sent to tech-kern@.

Rework the machine configuration interface.

Provide three ranges in the conf space: <libnvmm:0-100>, <MI:100-200> and
<MD:200-...>. Remove nvmm_callbacks_register(), and replace it by the conf
op NV

Rework the machine configuration interface.

Provide three ranges in the conf space: <libnvmm:0-100>, <MI:100-200> and
<MD:200-...>. Remove nvmm_callbacks_register(), and replace it by the conf
op NVMM_MACH_CONF_CALLBACKS, handled by libnvmm. The callbacks are now
per-machine, and the emulators should now do:

- nvmm_callbacks_register(&cbs);
+ nvmm_machine_configure(&mach, NVMM_MACH_CONF_CALLBACKS, &cbs);

This provides more granularity, for example if the process runs two VMs
and wants different callbacks for each.

show more ...

Modify the communication layer between the kernel NVMM driver and libnvmm:
introduce a bidirectionnal "comm page", a page of memory shared between
the kernel and userland, and used to transfer data i

Modify the communication layer between the kernel NVMM driver and libnvmm:
introduce a bidirectionnal "comm page", a page of memory shared between
the kernel and userland, and used to transfer data in and out in a more
performant manner than ioctls.

The comm page contains the VCPU state, plus three flags:

- "wanted": the states the kernel must get/set when requested via ioctls
- "cached": the states that are in the comm page
- "commit": the states the kernel must set in vcpu_run

The idea is to avoid performing expensive syscalls, by using the VCPU
state cached, either explicitly or speculatively, in the comm page. For
example, if the state is cached we do a direct 1->5 with no syscall:

+---------------------------------------------+
| Qemu |
+---------------------------------------------+
| ^
| (0) nvmm_vcpu_getstate | (6) Done
| |
V |
+---------------------------------------+
| libnvmm |
+---------------------------------------+
| ^ | ^
(1) State | | (2) No | (3) Ioctl: | (5) Ok, state
cached? | | | "please cache | fetched
| | | the state" |
V | | |
+-----------+ | |
| Comm Page |------+---------------+
+-----------+ |
^ |
(4) "Alright | V
babe" | +--------+
+-----| Kernel |
+--------+

The main changes in behavior are:

- nvmm_vcpu_getstate(): won't emit a syscall if the state is already
cached in the comm page, will just fetch from the comm page directly
- nvmm_vcpu_setstate(): won't emit a syscall at all, will just cache
the wanted state in the comm page
- nvmm_vcpu_run(): will commit the to-be-set state in the comm page,
as previously requested by nvmm_vcpu_setstate()

In addition to this, the kernel NVMM driver is changed to speculatively
cache certain states known to be of interest, so that the future
nvmm_vcpu_getstate() calls libnvmm or the emulator will perform will use
the comm page rather than expensive syscalls. For example, if an I/O
VMEXIT occurs, the I/O Assist in libnvmm will want GPRS+SEGS+CRS+MSRS,
and now the kernel caches all of that in the comm page before returning
to userland.

Overall, in a normal run of Windows 10, this saves several millions of
syscalls. Eg on a 4CPU Intel with 4VCPUs, booting the Win10 install ISO
goes from taking 1min35 to taking 1min16.

The libnvmm API is not changed, but the ABI is. If we changed the API it
would be possible to save expensive memcpys on libnvmm's side. This will
be avoided in a future version. The comm page can also be extended to
implement future services.

show more ...

Check the GPA permissions too in the Assists, because it is possible that
the guest traps on a page the virtualizer marked as read-only (even if it
appears as read-write in the HVA).

Micro optimizations:

- Compress x86_rexpref, x86_regmodrm, x86_opcode and x86_instr.
- Cache-align the register, opcode and group tables.
- Modify the opcode tables to have 256 entries, and avoid

Micro optimizations:

- Compress x86_rexpref, x86_regmodrm, x86_opcode and x86_instr.
- Cache-align the register, opcode and group tables.
- Modify the opcode tables to have 256 entries, and avoid a lookup.

show more ...

Change the layout of the SEG state:

- Reorder it, to match the CPU encoding. This is the universal order,
also used by Qemu. Drop the seg_to_nvmm[] tables.

- Compress it. This divides its size

Change the layout of the SEG state:

- Reorder it, to match the CPU encoding. This is the universal order,
also used by Qemu. Drop the seg_to_nvmm[] tables.

- Compress it. This divides its size by two.

- Rename some of its fields, to better match the x86 spec. Also, take S
out of Type, this was a NetBSD-ism that was likely confusing to other
people.

show more ...

Set hardseg to -1 rather than 0, because 0 can be a valid segment.

Fix handling of SIB instructions. We were jumping to the SIB node _before_
fetching the displacement, so the node would always think there was no
displacement.

This didn't alter the final GPA we wou

Fix handling of SIB instructions. We were jumping to the SIB node _before_
fetching the displacement, so the node would always think there was no
displacement.

This didn't alter the final GPA we would be touching - because it is
fetched from the kernel directly and not from the computation -, but it
altered the instruction length, and on some guests (like Fedora 64bit),
the VCPU would resume execution at the wrong RIP and crash.

Now these guests work.

show more ...

Remove the PSE check in the 32bit-PAE MMU. Setting CR4.PAE automatically
enables PSE regardless of whether CR4.PSE is set or not, so we should just
ignore it.

With this in place I can boot Windows 8

Remove the PSE check in the 32bit-PAE MMU. Setting CR4.PAE automatically
enables PSE regardless of whether CR4.PSE is set or not, so we should just
ignore it.

With this in place I can boot Windows 8.1 on NVMM.

show more ...

Harmonize the handling of the CPL between AMD and Intel.

AMD has a separate guest CPL field, because on AMD, the SYSCALL/SYSRET
instructions do not force SS.DPL to predefined values. On Intel they d

Harmonize the handling of the CPL between AMD and Intel.

AMD has a separate guest CPL field, because on AMD, the SYSCALL/SYSRET
instructions do not force SS.DPL to predefined values. On Intel they do,
so the CPL on Intel is just the guest's SS.DPL value.

Even though technically possible on AMD, there is no sane reason for a
guest kernel to set a non-three SS.DPL, doing that would mess up several
common segmentation practices and wouldn't be compatible with Intel.

So, force the Intel behavior on AMD, by always setting SS.DPL<=>CPL.
Remove the now unused CPL field from nvmm_x64_state::misc[]. This actually
increases performance on AMD: to detect interrupt windows the virtualizer
has to modify some fields of misc[], and because CPL was there, we had to
flush the SEG set of the VMCB cache. Now there is no flush necessary.

While here remove the CPL check for XSETBV on Intel, contrary to AMD
Intel checks the CPL before the intercept, so if we receive an XSETBV
VMEXIT, we are certain that it was executed at CPL=0 in the guest. By the
way my check was wrong in the first place, it was reading SS.RPL instead
of SS.DPL.

show more ...

Optimize: fetch only 5 bytes instead of 15, the instruction can have only
up to five prefixes.

#### is not legal.

Improvements:

- Emulate the instructions by executing them directly on the host CPU.
This is easier and probably faster than doing it in software
manually.

- Decode SUB from Primary, CMP fr

Improvements:

- Emulate the instructions by executing them directly on the host CPU.
This is easier and probably faster than doing it in software
manually.

- Decode SUB from Primary, CMP from Group1, TEST from Group3, and add
associated tests.

- Handle correctly the cases where an instruction that always implicitly
reads the register operand is executed with the mem operand as source
(eg: "orq (%rbx),%rax").

- Fix the MMU handling of 32bit-PAE. Under PAE CR3 is not page-aligned,
so there are extra bits that are valid.

With these changes in place I can boot Windows XP on Qemu+NVMM.

show more ...