nvmm(4): Add suspend/resume support.

New MD nvmm_impl callbacks:

- .suspend_interrupt forces all VMs on all physical CPUs to exit.
- .vcpu_suspend suspends an individual vCPU on a machine.
- .machi

nvmm(4): Add suspend/resume support.

New MD nvmm_impl callbacks:

- .suspend_interrupt forces all VMs on all physical CPUs to exit.
- .vcpu_suspend suspends an individual vCPU on a machine.
- .machine_suspend suspends an individual machine.
- .suspend suspends the whole system.
- .resume resumes the whole system.
- .machine_resume resumes an individual machine.
- .vcpu_resume resumes an indidivudal vCPU on a machine.

Suspending nvmm:

1. causes new VM operations (ioctl and close) to block until resumed,
2. uses .suspend_interrupt to interrupt any concurrent and force them
to return early, and then
3. uses the various suspend callbacks to suspend all vCPUs, machines,
and the whole system -- all vCPUs before the machine they're on,
and all machines before the system.

Resuming nvmm does the reverse of (3) -- resume system, resume each
machine and then the vCPUs on that machine -- and then unblocks
operations.

Implemented only for x86-vmx for now:

- suspend_interrupt triggers a TLB IPI to cause VM exits;
- vcpu_suspend issues VMCLEAR to force any in-CPU state to be written
to memory;
- machine_suspend does nothing;
- suspend does VMXOFF on all CPUs;
- resume does VMXON on all CPUs;
- machine_resume does nothing; and
- vcpu_resume just marks each vCPU as valid but inactive so
subsequent use will clear it and load it with vmptrld.

x86-svm left as an exercise for the reader.

show more ...

Only detach the cfdriver if we just attached it.

Report errno in message when fail to attach cdevsw

nvmm(4): Fix typo in previous.

Didn't trip over this in my test build because nvmm is a module,
oops.

uvm(9): fo_mmap caller guarantees positive size.

No functional change intended, just sprinkling assertions to make it
clearer.

be sure to only access vcpu if it was initialised.

Implement nvmm_vcpu::stop, a race-free exit from nvmm_vcpu_run() without
signals. This introduces a new kernel and userland NVMM version indicating
this support.

Patch by Kamil Rytarowski <kamil@net

Implement nvmm_vcpu::stop, a race-free exit from nvmm_vcpu_run() without
signals. This introduces a new kernel and userland NVMM version indicating
this support.

Patch by Kamil Rytarowski <kamil@netbsd.org> and committed on his request.

show more ...

nvmm: cosmetic changes

- Style.
- Explicitly include ioccom.h.

Round of uvm.h cleanup.

The poorly named uvm.h is generally supposed to be for uvm-internal
users only.

- Narrow it to files that actually need it -- mostly files that need
to query whether curlw

Round of uvm.h cleanup.

The poorly named uvm.h is generally supposed to be for uvm-internal
users only.

- Narrow it to files that actually need it -- mostly files that need
to query whether curlwp is the pagedaemon, which should maybe be
exposed by an external header.

- Use uvm_extern.h where feasible and uvm_*.h for things not exposed
by it. We should split up uvm_extern.h but this will serve for now
to reduce the uvm.h dependencies.

- Use uvm_stat.h and #ifdef UVMHIST uvm.h for files that use
UVMHIST(ubchist), since ubchist is declared in uvm.h but the
reference evaporates if UVMHIST is not defined, so we reduce header
file dependencies.

- Make uvm_device.h and uvm_swap.h independently includable while
here.

ok chs@

show more ...

nvmm: update copyright headers

nvmm: more __read_mostly

nvmm: explicitly include atomic.h

nvmm: misc improvements

- use mach->ncpus to get the number of vcpus, now that we have it
- don't forget to decrement mach->ncpus when a machine gets killed
- add more __predict_false()

nvmm: use relaxed atomics to read nmachines

nvmm: localify a variable that doesn't need to be global

Put the few x86-specific structures under #ifdef __x86_64__, for clarity.

Print the backend name when attaching.

Register NVMM as an actual pseudo-device. Without PMF handler, to
explicitly disallow ACPI suspend if NVMM is running.

Should fix PR/55406.

Gather the conditions to return from the VCPU loops in nvmm_return_needed(),
and use it in nvmm_do_vcpu_run() as well. This fixes two undesired behaviors:

- When a VM initializes, the many nested p

Gather the conditions to return from the VCPU loops in nvmm_return_needed(),
and use it in nvmm_do_vcpu_run() as well. This fixes two undesired behaviors:

- When a VM initializes, the many nested page faults that need processing
could cause the calling thread to occupy the CPU too much if we're unlucky
and are only getting repeated nested page faults thousands of times in a
row.

- When the emulator calls nvmm_vcpu_run() and immediately sends a signal to
stop the VCPU, it's better to check signals earlier and leave right away,
rather than doing a round of VCPU run that could increase the time spent
by the emulator waiting for the return.

show more ...

Complete rev1.26: reset nvmm_impl to NULL in nvmm_fini().

On Intel CPUs, CPUID leaf 0xB, too, provides topology information, so
filter it correctly, to avoid inconsistencies if the host has SMT.

This fixes HaikuOS which fetches SMT information from there a

On Intel CPUs, CPUID leaf 0xB, too, provides topology information, so
filter it correctly, to avoid inconsistencies if the host has SMT.

This fixes HaikuOS which fetches SMT information from there and would
panic because of the inconsistencies.

show more ...

When the identification fails, print the reason.

In nvmm_open(), make sure an implementation was found. This fixes an
initialization bug triggerable in certain conditions.

If you build nvmm inside the kernel, AND have a cpu that is not supported,

In nvmm_open(), make sure an implementation was found. This fixes an
initialization bug triggerable in certain conditions.

If you build nvmm inside the kernel, AND have a cpu that is not supported,
AND run nvmmctl (or qemu-nvmm, both being the only binaries in the "nvmm"
group), you get a page fault.

This is because when nvmm is built inside the kernel, the kernel registers
nvmm_cdevsw behind nvmm's back. The ioctl is therefore always accessible,
and will hit NULL pointers if nvmm_init() failed.

Problem reported by Andrei M. on netbsd-users@, thanks.

show more ...

Add nram in struct nvmm_ctl_mach_info.

Change the way root_owner works: consider the calling process as root_owner
not if it has root privileges, but if the /dev/nvmm device was opened with
write permissions. Introduce the undocumented nv

Change the way root_owner works: consider the calling process as root_owner
not if it has root privileges, but if the /dev/nvmm device was opened with
write permissions. Introduce the undocumented nvmm_root_init() function to
achieve that.

The goal is to simplify the logic and have more granularity, eg if we want
a monitoring agent to access VMs but don't want to give this agent real
root access on the system.

show more ...

Miscellaneous changes in NVMM, to address several inconsistencies and
issues in the libnvmm API.

- Rename NVMM_CAPABILITY_VERSION to NVMM_KERN_VERSION, and check it in
libnvmm. Introduce NVMM_US

Miscellaneous changes in NVMM, to address several inconsistencies and
issues in the libnvmm API.

- Rename NVMM_CAPABILITY_VERSION to NVMM_KERN_VERSION, and check it in
libnvmm. Introduce NVMM_USER_VERSION, for future use.

- In libnvmm, open "/dev/nvmm" as read-only and with O_CLOEXEC. This is to
avoid sharing the VMs with the children if the process forks. In the
NVMM driver, force O_CLOEXEC on open().

- Rename the following things for consistency:
nvmm_exit* -> nvmm_vcpu_exit*
nvmm_event* -> nvmm_vcpu_event*
NVMM_EXIT_* -> NVMM_VCPU_EXIT_*
NVMM_EVENT_INTERRUPT_HW -> NVMM_VCPU_EVENT_INTR
NVMM_EVENT_EXCEPTION -> NVMM_VCPU_EVENT_EXCP
Delete NVMM_EVENT_INTERRUPT_SW, unused already.

- Slightly reorganize the MI/MD definitions, for internal clarity.

- Split NVMM_VCPU_EXIT_MSR in two: NVMM_VCPU_EXIT_{RD,WR}MSR. Also provide
separate u.rdmsr and u.wrmsr fields. This is more consistent with the
other exit reasons.

- Change the types of several variables:
event.type enum -> u_int
event.vector uint64_t -> uint8_t
exit.u.*msr.msr: uint64_t -> uint32_t
exit.u.io.type: enum -> bool
exit.u.io.seg: int -> int8_t
cap.arch.mxcsr_mask: uint64_t -> uint32_t
cap.arch.conf_cpuid_maxops: uint64_t -> uint32_t

- Delete NVMM_VCPU_EXIT_MWAIT_COND, it is AMD-only and confusing, and we
already intercept 'monitor' so it is never armed.

- Introduce vmx_exit_insn() for NVMM-Intel, similar to svm_exit_insn().
The 'npc' field wasn't getting filled properly during certain VMEXITs.

- Introduce nvmm_vcpu_configure(). Similar to nvmm_machine_configure(),
but as its name indicates, the configuration is per-VCPU and not per-VM.
Migrate and rename NVMM_MACH_CONF_X86_CPUID to NVMM_VCPU_CONF_CPUID.
This becomes per-VCPU, which makes more sense than per-VM.

- Extend the NVMM_VCPU_CONF_CPUID conf to allow triggering VMEXITs on
specific leaves. Until now we could only mask the leaves. An uint32_t
is added in the structure:
uint32_t mask:1;
uint32_t exit:1;
uint32_t rsvd:30;
The two first bits select the desired behavior on the leaf. Specifying
zero on both resets the leaf to the default behavior. The new
NVMM_VCPU_EXIT_CPUID exit reason is added.

show more ...