kernel - Remove lwkt_msg from sysunion in syscall code

* The lmsg field (lwkt_msg) is no longer used in the
sysunion union, remove it. Should not result in any
material changes to code generati

kernel - Remove lwkt_msg from sysunion in syscall code

* The lmsg field (lwkt_msg) is no longer used in the
sysunion union, remove it. Should not result in any
material changes to code generation.

show more ...

kernel - Flesh out Spectre mitigation support

* Add handling for all modes for AMD CPUs, including support for
IBRS_AUTO and STIBP_AUTO (always-on) bits which will be added
to future cpus.

* Ad

kernel - Flesh out Spectre mitigation support

* Add handling for all modes for AMD CPUs, including support for
IBRS_AUTO and STIBP_AUTO (always-on) bits which will be added
to future cpus.

* Add STIBP handling to Intel CPUs. I can't find definitions for
AUTO (always-on) modes for Intel, so those are still not supported
(no current CPU has AUTO support yet anyway).

* Current DragonFlyBSD defaults: Will enable IBRS_AUTO and STIBP_AUTO
(always on) support by default if the cpu has it. Will NOT enable IBRS
or STIBP (non-auto) toggling by default. Will not enable IBPB by default.

IBPB is currently not enabled by default. The overhead is an enormous
~2uS. We will follow Linux in this regard.

* Change the machdep.spectre_mitigation sysctl to take a string of
features to enable. Change machdep.spectre_support to display a
string of features supported. Possible features are:

IBRS Indirect Branch Restricted Speculation (U->K and K->U)
STIBP Single Thread Indirect Branch Prediction (U->K and K->U)
IBPB Branch Prediction Barrier (U->K)
IBRS_AUTO IBRS always-on (set once and forget)
STIBP_AUTO STIBP always-on (set once and forget)

The machdep.spectre_support sysctl tells you whats available.

* Refactor tr_pcb_gflags into tr_pcb_spec_ctrl[2] to make it easier
for the assembly code to program the SPEC_CTRL MSR.

* Note that some of the above bits will never be supported by current
hardware and exist to allow future hardware to support these features
in a less expensive manner.

* Also note that for Meltdown, AMD is immune and the meltdown mitigation
will not be enabled. Intel is vulnerable and the mitigation will be
enabled by default. See sysctl machdep.meltdown_mitigation.

show more ...

kernel - Fix CVE-2018-8897, debug register issue

* #DB can be delayed in a way that causes it to occur on the first
instruction of the int $3 or syscall handlers. These handlers must
be able to

kernel - Fix CVE-2018-8897, debug register issue

* #DB can be delayed in a way that causes it to occur on the first
instruction of the int $3 or syscall handlers. These handlers must
be able to detect and handle the condition. This is a historical
artifact of cpu operation that has existed for a very long time on
both AMD and Intel CPUs.

* Fix by giving #DB its own trampoline stack and a way to load a
deterministic %gs and %cr3 independent of the normal CS check.
This is CVE-2018-8897.

* Also fix the NMI trampoline while I'm here.

* Also fix an old issue with debug register trace traps which can
occur when the kernel is accessing the user's address space.
This fix was lost years ago, now recovered.

Credits: Nick Peterson of Everdox Tech, LLC (original reporter)
Credits: Thanks to Microsoft for coordinating the OS vendor response

show more ...

kernel - Implement spectre mitigations part 1

* Implement machdep.spectre_mitigation. This can be set as a tunable
or sysctl'd later. The tunable is only applicable if the BIOS has
the appropr

kernel - Implement spectre mitigations part 1

* Implement machdep.spectre_mitigation. This can be set as a tunable
or sysctl'd later. The tunable is only applicable if the BIOS has
the appropriate microcode, otherwise you have to update the microcode
first and then use sysctl to set the mode.

This works similarly to Linux's IBRS support.

mode 0 - Spectre IBPB MSRs disabled

mode 1 - Sets IBPB MSR on USER->KERN transition and clear it
on KERN->USER.

mode 2 - Leave IBPB set globally. Do not toggle on USER->KERN or
KERN->USER transitions.

* Retest spectre microcode MSRs on microcode update.

* Spectre mode 1 is enabled by default if the microcode supports it.
(we might change this to disabled by default, I'm still mulling it
over).

* General performance effects (not counting the MMU separation mode,
which is machdep.meltdown_mitigation and adds another 3% in overhead):

Skylake loses around 5% for mode 1 and 12% for mode 2, verses mode 0.
Haswell loses around 12% for mode 1 and 53% for mode 2, verses mode 0.

Add another 3% if MMU separation is also turned on (aka
machdep.meltdown_mitigation).

* General system call overhead effects on Skylake:

machdep.meltdown_mitigation=0, machdep.spectre_mitigation=0 103ns
machdep.meltdown_mitigation=1, machdep.spectre_mitigation=0 360ns
machdep.meltdown_mitigation=1, machdep.spectre_mitigation=1 848ns
machdep.meltdown_mitigation=1, machdep.spectre_mitigation=2 404ns

Note that mode 1 has better overall performance for mixed user+kernel
workloads despite having a much higher system call overhead, whereas
mode 2 has lower system call overhead but generally lower overall
performance because IBPB is enabled in usermode.

show more ...

kernel - Intel user/kernel separation MMU bug fix part 3/3

* Implement the isolated pmap template, iso_pmap. The pmap code will
generate a dummy iso_pmap containing only the kernel mappings requi

kernel - Intel user/kernel separation MMU bug fix part 3/3

* Implement the isolated pmap template, iso_pmap. The pmap code will
generate a dummy iso_pmap containing only the kernel mappings required
for userland to be able to transition into the kernel and vise-versa.

The mappings needed are:

(1) The per-cpu trampoline area for our stack (rsp0)
(2) The global descriptor table (gdt) for all cpus
(3) The interrupt descriptor table (idt) for all cpus
(4) The TSS block for all cpus (we store this in the trampoline page)
(5) Kernel code addresses for the interrupt vector entry and exit

* In this implementation the 'kernel code' addresses are currently just
btext to etext. That is, the kernel's primary text area. Kernel
data and bss are not part of the isolation map.

TODO - just put the vector entry and exit points in the map, and
not the entire kernel.

* System call performance is reduced when isolation is turned on.
100ns -> 350ns or so. However, typical workloads should not lose
more than 5% performance or so. System-call heavy and interrupt-heavy
workloads (network, database, high-speed storage, etc) can lose a lot
more performance.

We leave the trampoline code in-place whether isolation is turned on
or not. The trampoline overhead, without isolation, is only 5nS or so.

* Fix a missing exec-related trampoline initialization.

* Clean-up kernel page table PTEs a bit. PG_M is ignored on non-terminal
PTEs, so don't set it. Also don't set PG_U in non-terminal kernel
page table pages (PG_U is never set on terminal PTEs so this wasn't
a problem, but we should be correct).

* Fix a bug in fast_syscall's trampoline stack. The wrong stack
pointer was being loaded.

* Move mdglobaldata->gd_common_tss to privatespace->common_tss.
Place common_tss in the same page as the trampoline to reduce
exposure to globaldata from the isolated MMU context.

* 16-byte align struct trampframe for convenience.

* Fix a bug in POP_FRAME. Always cli in order to avoid getting
an interrupt just at the iretq instruction, which might be
misinterpreted.

show more ...

kernel - Intel user/kernel separation MMU bug fix part 2/3

* Cleanup pass. Throw in some documentation.

* Move the gd_pcb_* fields into the trampoline page to allow
kernel memory to be further r

kernel - Intel user/kernel separation MMU bug fix part 2/3

* Cleanup pass. Throw in some documentation.

* Move the gd_pcb_* fields into the trampoline page to allow
kernel memory to be further restricted in part 3.

show more ...

kernel - Intel user/kernel separation MMU bug fix part 1/3

* Part 1/3 of the fix for the Intel user/kernel separation MMU bug.
It appears that it is possible to discern the contents of kernel
me

kernel - Intel user/kernel separation MMU bug fix part 1/3

* Part 1/3 of the fix for the Intel user/kernel separation MMU bug.
It appears that it is possible to discern the contents of kernel
memory with careful timing measurements of instructions due to
speculative memory reads and speculative instruction execution
by Intel cpus. This can happen because Intel will allow both to
occur even when the memory access is later disallowed due to
privilege separation in the PTE.

Even though the execution is always aborted, the speculative
reads and speculative execution results in timing artifacts which
can be measured. A speculative compare/branch can lead to timing
artifacts that allow the actual contents of kernel memory to be
discerned.

While there are multiple speculative attacks possible, the Intel
bug is particularly bad because it allows a user program to more
or less effortlessly access kernel memory (and if a DMAP is
present, all of physical memory).

* Part 1 implements all the logic required to load an 'isolated'
version of the user process's PML4e into %cr3 on all user
transitions, and to load the 'normal' U+K version into %cr3 on
all transitions from user to kernel.

* Part 1 fully allocates, copies, and implements the %cr3 loads for
the 'isolated' version of the user process PML4e.

* Part 1 does not yet actually adjust the contents of this isolated
version to replace the kernel map with just a trampoline map in
kernel space. It does remove the DMAP as a test, though. The
full separation will be done in part 3.

show more ...

Remove advertising header from sys/

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Some less clear cases taken as it was done of FreeBSD.

hammer2 - Merge Mihai Carabas's VKERNEL/VMM GSOC project into the main tree

* This merge contains work primarily by Mihai Carabas, with some misc
fixes also by Matthew Dillon.

* Special note on G

hammer2 - Merge Mihai Carabas's VKERNEL/VMM GSOC project into the main tree

* This merge contains work primarily by Mihai Carabas, with some misc
fixes also by Matthew Dillon.

* Special note on GSOC core

This is, needless to say, a huge amount of work compressed down into a
few paragraphs of comments. Adds the pc64/vmm subdirectory and tons
of stuff to support hardware virtualization in guest-user mode, plus
the ability for programs (vkernels) running in this mode to make normal
system calls to the host.

* Add system call infrastructure for VMM mode operations in kern/sys_vmm.c
which vectors through a structure to machine-specific implementations.

vmm_guest_ctl_args()
vmm_guest_sync_addr_args()

vmm_guest_ctl_args() - bootstrap VMM and EPT modes. Copydown the original
user stack for EPT (since EPT 'physical' addresses cannot reach that far
into the backing store represented by the process's original VM space).
Also installs the GUEST_CR3 for the guest using parameters supplied by
the guest.

vmm_guest_sync_addr_args() - A host helper function that the vkernel can
use to invalidate page tables on multiple real cpus. This is a lot more
efficient than having the vkernel try to do it itself with IPI signals
via cpusync*().

* Add Intel VMX support to the host infrastructure. Again, tons of work
compressed down into a one paragraph commit message. Intel VMX support
added. AMD SVM support is not part of this GSOC and not yet supported
by DragonFly.

* Remove PG_* defines for PTE's and related mmu operations. Replace with
a table lookup so the same pmap code can be used for normal page tables
and also EPT tables.

* Also include X86_PG_V defines specific to normal page tables for a few
situations outside the pmap code.

* Adjust DDB to disassemble SVM related (intel) instructions.

* Add infrastructure to exit1() to deal related structures.

* Optimize pfind() and pfindn() to remove the global token when looking
up the current process's PID (Matt)

* Add support for EPT (double layer page tables). This primarily required
adjusting the pmap code to use a table lookup to get the PG_* bits.

Add an indirect vector for copyin, copyout, and other user address space
copy operations to support manual walks when EPT is in use.

A multitude of system calls which manually looked up user addresses via
the vm_map now need a VMM layer call to translate EPT.

* Remove the MP lock from trapsignal() use cases in trap().

* (Matt) Add pthread_yield()s in most spin loops to help situations where
the vkernel is running on more cpu's than the host has, and to help with
scheduler edge cases on the host.

* (Matt) Add a pmap_fault_page_quick() infrastructure that vm_fault_page()
uses to try to shortcut operations and avoid locks. Implement it for
pc64. This function checks whether the page is already faulted in as
requested by looking up the PTE. If not it returns NULL and the full
blown vm_fault_page() code continues running.

* (Matt) Remove the MP lock from most the vkernel's trap() code

* (Matt) Use a shared spinlock when possible for certain critical paths
related to the copyin/copyout path.

show more ...

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

rename amd64 architecture to x86_64

The rest of the world seems to call amd64 x86_64. Bite the bullet and
rename all of the architecture files and references. This will
hopefully make pkgsrc build

rename amd64 architecture to x86_64

The rest of the world seems to call amd64 x86_64. Bite the bullet and
rename all of the architecture files and references. This will
hopefully make pkgsrc builds less painful.

Discussed-with: dillon@

show more ...

rename amd64 architecture to x86_64

The rest of the world seems to call amd64 x86_64. Bite the bullet and
rename all of the architecture files and references. This will
hopefully make pkgsrc build

rename amd64 architecture to x86_64

The rest of the world seems to call amd64 x86_64. Bite the bullet and
rename all of the architecture files and references. This will
hopefully make pkgsrc builds less painful.

Discussed-with: dillon@

show more ...

kernel - x86_64 - Remove intrframe->if_gs

* The if_gs field doesn't exist in the actual trap frame.

amd64: Update some header files for the vkernel.