Update register name at fault. Pointed out by hgutch@n.o, thanks.

In vetor.S:r1.78
====================
IDTVEC(handle_hypervisor_callback)
movl CPUVAR(ILEVEL),%edi
pushq %rdi /* fo

Update register name at fault. Pointed out by hgutch@n.o, thanks.

In vetor.S:r1.78
====================
IDTVEC(handle_hypervisor_callback)
movl CPUVAR(ILEVEL),%edi
pushq %rdi /* for Xdoreti */
====================
So, use %edi as before.

show more ...

NetBSD/x86: Raise the number of interrupt sources per CPU from 32 to 56.

There has been no objection for three years.
https://mail-index.netbsd.org/port-amd64/2019/09/22/msg003012.html
Implement

NetBSD/x86: Raise the number of interrupt sources per CPU from 32 to 56.

There has been no objection for three years.
https://mail-index.netbsd.org/port-amd64/2019/09/22/msg003012.html
Implemented by nonaka@n.o, updated by me.

show more ...

Some devices (e.g. ixg in MSI-X mode) don't to have their handlers called
when no interrupt are pending. So add an extra ih_pending field
to struct intrhand, which is incremeted when the handler is n

Some devices (e.g. ixg in MSI-X mode) don't to have their handlers called
when no interrupt are pending. So add an extra ih_pending field
to struct intrhand, which is incremeted when the handler is not called because
of IPL level and reset to 0 when called. Check this in Xen's resume
assembly to call only handlers that are really pending.

show more ...

Flag EOI on TLB shootdown IPIs after the shootdown has been processed.

Explicitly align to 8 bytes, found by kUBSan.

Reported-by: syzbot+f1e1561ed739db869d44@syzkaller.appspotmail.com

Limit the amount of code in TEXT_USER_BEGIN/TEXT_USER_END for
hypervisor_pvhvm_callback, for the benefit of SVS. Suggested by maxv@

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

Fix a problem with intr_unmask() that can cause a forever-loop:
- When handling the source-is-masked case in the interrupt vector, set the
interrupt bit in a new ci_imasked field and ensure the bit

Fix a problem with intr_unmask() that can cause a forever-loop:
- When handling the source-is-masked case in the interrupt vector, set the
interrupt bit in a new ci_imasked field and ensure the bit is cleared
from ci_ipending.
- In intr_unmask(), transfer the bit from ci_imasked to ci_ipending for
non-level-sensitive interrupts (the PIC does the work for us in the
level-sensitive case), and only force pending interrupts to be processed
in this case. (In all cases, make sure the now-unmasked bit is cleared
from ci_imasked.)

Before, the bit was left in ci_ipending so as not to use edge-triggered
interrupts while the source is masked, but Xspllower() relies on the
pending bits getting cleared.

Tested by forcing all wm(4) interrupts on my test system though an
intr_mask() / softint / intr_unmask() cycle and exercising the network
heavily.

show more ...

Add intr_mask() and corresponding intr_unmask() calls that allow specific
interrupt lines / sources to be masked as needed (rather than making a
set of sources by IPL as with spl*()).

Disable KCOV - by raising the interrupt level - in the TLB IPI handler,
because this is only noise.

Use IDTVEC instead of NENTRY for handle_hyperv_hypercall.

Added Microsoft Hyper-V support. It ported from OpenBSD and FreeBSD.

graphical console is not work on Gen.2 VM yet. To use the serial console,
enter "consdev com,0x3f8,115200" on efiboot.

Welcome XENPVHVM mode.

It is UP only, has xbd(4) and xennet(4) as PV drivers.

The console is com0 at isa and the native portion is very
rudimentary AT architecture, so is probably suboptimal to
run

Welcome XENPVHVM mode.

It is UP only, has xbd(4) and xennet(4) as PV drivers.

The console is com0 at isa and the native portion is very
rudimentary AT architecture, so is probably suboptimal to
run without PV support.

show more ...

In preparation for debut-ing PVHVM mode:

- Make the struct intrstub uniform across native and XEN.
- Introduce vector callback entrypoints for PVHVM mode.

We reorganise definitions for XEN source support as follows:

XEN - common sources required for baseline XEN support.
XENPV - sources required for support of XEN in PV mode.
XENPVHVM - sources requir

We reorganise definitions for XEN source support as follows:

XEN - common sources required for baseline XEN support.
XENPV - sources required for support of XEN in PV mode.
XENPVHVM - sources required for support for XEN in HVM mode.
XENPVH - sources required for support for XEN in PVH mode.

show more ...

Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exportin

Excise XEN specific code out of x86/x86/intr.c into xen/x86/xen_intr.c

While at it, separate the source function tracking so that the interrupt
paths are truly independant.

Use weak symbol exporting to provision for future PVHVM co-existence
of both files, but with independant paths. Introduce assembler code
such that in a unified scenario, native interrupts get first priority
in spllower(), followed by XEN event callbacks. IPL management and
semantics are unchanged - native handlers and xen callbacks are
expected to maintain their ipl related semantics.

In summary, after this commit, native and XEN now have completely
unrelated interrupt handling mechanisms, including
intr_establish_xname() and assembler stubs and intr handler
management.

Happy Christmas!

show more ...

Drop NENTRY() from the x86 kernels, use ENTRY(). With PMCs (and other hardware
tracing facilities) we have a much better ways of monitoring the CPU activity
than GPROF, without software modification.

Drop NENTRY() from the x86 kernels, use ENTRY(). With PMCs (and other hardware
tracing facilities) we have a much better ways of monitoring the CPU activity
than GPROF, without software modification.

Also I think GPROF has never worked, because the 'start' functions of both
i386 and amd64 use ENTRY(), and it would have caused a function call while the
kernel was not yet relocated.

show more ...

Put these arrays in .rodata, they aren't supposed to be executable.

Rename the DDB IPI IDT vectors for consistency. ok maxv@

Rename "handle_" -> "Xhandle_", and add the function names (introduced by
SVS) in db_machdep.c.

Should fix the DDB part of PR/53060.

Define legacy_stubs in a macro.

Rename i8259_stubs -> legacy_stubs. We will want the entries to have the
same name, eg:

legacy_stubs
-> Xintr_legacy0, Xrecurse_legacy0, Xresume_legacy0
-> Xintr_legacy1, Xrecurse_legacy1, Xres

Rename i8259_stubs -> legacy_stubs. We will want the entries to have the
same name, eg:

legacy_stubs
-> Xintr_legacy0, Xrecurse_legacy0, Xresume_legacy0
-> Xintr_legacy1, Xrecurse_legacy1, Xresume_legacy1
...

show more ...

Define INTRSTUB_ARRAY, simplifies a lot.

Declare and use INTR_RECURSE_ENTRY, an optimized version of INTRENTRY.
When processing deferred interrupts, we are always entering the new
handler in kernel mode, so there is no point performing the

Declare and use INTR_RECURSE_ENTRY, an optimized version of INTRENTRY.
When processing deferred interrupts, we are always entering the new
handler in kernel mode, so there is no point performing the userland
checks.

Saves several instructions.

show more ...

Unmap the kernel from userland in SVS, and leave only the needed
trampolines. As explained below, SVS should now completely mitigate
Meltdown on GENERIC kernels, even though it needs some more tweaki

Unmap the kernel from userland in SVS, and leave only the needed
trampolines. As explained below, SVS should now completely mitigate
Meltdown on GENERIC kernels, even though it needs some more tweaking
for GENERIC_KASLR.

Until now the kernel entry points looked like:

FUNC(intr)
pushq $ERR
pushq $TRAPNO
INTRENTRY
... handle interrupt ...
INTRFASTEXIT
END(intr)

With this change they are split and become:

FUNC(handle)
... handle interrupt ...
INTRFASTEXIT
END(handle)

TEXT_USER_BEGIN
FUNC(intr)
pushq $ERR
pushq $TRAPNO
INTRENTRY
jmp handle
END(intr)
TEXT_USER_END

A new section is introduced, .text.user, that contains minimal kernel
entry/exit points. In order to choose what to put in this section, two
macros are introduced, TEXT_USER_BEGIN and TEXT_USER_END.

The section is mapped in userland with normal 4K pages.

In GENERIC, the section is 4K-page-aligned and embedded in .text, which
is mapped with large pages. That is to say, when an interrupt comes in,
the CPU has the user page tables loaded and executes the 'intr' functions
on 4K pages; after calling SVS_ENTER (in INTRENTRY) these 4K pages become
2MB large pages, and remain so when executing in kernel mode.

In GENERIC_KASLR, the section is 4K-page-aligned and independent from the
other kernel texts. The prekern just picks it up and maps it at a random
address.

In GENERIC, SVS should now completely mitigate Meltdown: what we put in
.text.user is not secret.

In GENERIC_KASLR, SVS would have to be improved a bit more: the
'jmp handle' instruction is actually secret, since it leaks the address
of the section we are jumping into. By exploiting Meltdown on Intel, this
theoretically allows a local user to reconstruct the address of the first
text section. But given that our KASLR produces several texts, and that
each section is not correlated with the others, the level of protection
KASLR provides is still good.

show more ...