thread: Add a return value to cpu_set_upcall()

Some implementations copy data to userspace, an operation which can in
principle fail. In preparation for adding a __result_use_check
annotation to co

thread: Add a return value to cpu_set_upcall()

Some implementations copy data to userspace, an operation which can in
principle fail. In preparation for adding a __result_use_check
annotation to copyin() and related functions, let implementations of
cpu_set_upcall() return an error, and check for errors when copying data
to user memory.

Reviewed by: kib, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43100

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

kmem_malloc/free: Use void * instead of vm_offset_t for kernel pointers.

Reviewed by: kib, markj
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D36549

amd64: expicitly re-init td_frame in copy_thread()

Otherwise we are using whatever the value was left from the previous
thread run on kernel entry from usermode. Typically it would be the
desired va

amd64: expicitly re-init td_frame in copy_thread()

Otherwise we are using whatever the value was left from the previous
thread run on kernel entry from usermode. Typically it would be the
desired value as is, but it is not guaranteed.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35888

show more ...

Remove dead code.

is_physical_memory() dead since 235a54de.

Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D35056
MFC after: 2 weeks

Simplify swi for bus_dma.

When a DMA request using bounce pages completes, a swi is triggered to
schedule pending DMA requests using the just-freed bounce pages. For
a long time this bus_dma swi ha

Simplify swi for bus_dma.

When a DMA request using bounce pages completes, a swi is triggered to
schedule pending DMA requests using the just-freed bounce pages. For
a long time this bus_dma swi has been tied to a "virtual memory" swi
(swi_vm). However, all of the swi_vm implementations are the same and
consist of checking a flag (busdma_swi_pending) which is always true
and if set calling busdma_swi. I suspect this dates back to the
pre-SMPng days and that the intention was for swi_vm to serve as a
mux. However, in the current scheme there's no need for the mux.

Instead, remove swi_vm and vm_ih. Each bus_dma implementation that
uses bounce pages is responsible for creating its own swi (busdma_ih)
which it now schedules directly. This swi invokes busdma_swi directly
removing the need for busdma_swi_pending.

One consequence is that the swi now works on RISC-V which had previously
failed to invoke busdma_swi from swi_vm.

Reviewed by: imp, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D33447

show more ...

amd64: Avoid copying td_frame from kernel procs

When creating a new thread, we unconditionally copy td_frame from the
creating thread. For threads which never return to user mode, this is
unnecessa

amd64: Avoid copying td_frame from kernel procs

When creating a new thread, we unconditionally copy td_frame from the
creating thread. For threads which never return to user mode, this is
unnecessary since td_frame just points to the base of the stack or a
random interrupt frame.

If KASAN is configured this copying may also trigger false positives
since the td_frame region may contain poisoned stack regions. It was
not noticed before since thread0 used a dummy proc0_tf trapframe, and
kernel procs are generally created by thread0. Since commit
df8dd6025af88a99d34f549fa9591a9b8f9b75b1, though, we call
cpu_thread_alloc(&thread0) when initializing FPU state, which
reinitializes thread0.td_frame.

Work around the problem by not copying the frame unless the copying
thread came from user mode. While here, de-duplicate the copying and
remove redundant re(initialization) of td_frame.

Reported by: syzbot+2ec89312bffbf38d9aec@syzkaller.appspotmail.com
Reviewed by: kib
Fixes: df8dd6025af8
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32057

show more ...

amd64: eliminate td_md.md_fpu_scratch

For signal send, copyout from the user FPU save area directly.

For sigreturn, we are in sleepable context and can do temporal
allocation of the transient save

amd64: eliminate td_md.md_fpu_scratch

For signal send, copyout from the user FPU save area directly.

For sigreturn, we are in sleepable context and can do temporal
allocation of the transient save area. We cannot copying from userspace
directly to user save area because XSAVE state needs to be validated,
also partial copyins can corrupt it.

Requested by: jhb
Reviewed by: jhb, markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31954

show more ...

amd64: stop using top of the thread' kernel stack for FPU user save area

Instead do one more allocation at the thread creation time. This frees
a lot of space on the stack.

Also do not use alloca(

amd64: stop using top of the thread' kernel stack for FPU user save area

Instead do one more allocation at the thread creation time. This frees
a lot of space on the stack.

Also do not use alloca() for temporal storage in signal delivery sendsig()
function and signal return syscall sys_sigreturn(). This saves equal
amount of space, again by the cost of one more allocation at the thread
creation time.

A useful experiment now would be to reduce KSTACK_PAGES.

Reviewed by: jhb, markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31954

show more ...

fork: Allow ABI to specify fork return values for child.

At least Linux x86 ABI's does not use carry bit and expects that the dx register
is preserved. For this add a new sv_set_fork_retval hook and

fork: Allow ABI to specify fork return values for child.

At least Linux x86 ABI's does not use carry bit and expects that the dx register
is preserved. For this add a new sv_set_fork_retval hook and call it from cpu_fork().

Add a short comment about touching dx in x86_set_fork_retval(), for more details
see phab comments from kib@ and imp@.

Reviewed by: kib
Differential revision: https://reviews.freebsd.org/D31472
MFC after: 2 weeks

show more ...

amd64: Clear the local TSS when creating a new thread

Otherwise it is copied from the creating thread. Then, if either thread
exits, the other is left with a dangling pointer, typically resulting i

amd64: Clear the local TSS when creating a new thread

Otherwise it is copied from the creating thread. Then, if either thread
exits, the other is left with a dangling pointer, typically resulting in
a page fault upon the next context switch.

Reported by: syzkaller
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30607

show more ...

x86: use x86_clear_dbregs() on fork

instead of manual zeroing of the debug registers file in pcb.
This centralizes the cleaning code, but the practical difference is
that PCB_DBREGS flag is cleared,

x86: use x86_clear_dbregs() on fork

instead of manual zeroing of the debug registers file in pcb.
This centralizes the cleaning code, but the practical difference is
that PCB_DBREGS flag is cleared, saving some operations on context
switching.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29687

show more ...

x86: clear %db registers in new process

Reported by: Michał Górny <mgorny@gentoo.org>
PR: 254661
Reviewed by: emaste, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential revisio

x86: clear %db registers in new process

Reported by: Michał Górny <mgorny@gentoo.org>
PR: 254661
Reviewed by: emaste, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D29496

show more ...

Rename linux_set_upcall_kse() to linux_set_upcall().

This matches the rename of cpu_set_upcall_kse() in
5c2cf818454375536fda522ba83cf67c50929e6b.

Reviewed by: kib, emaste
MFC after: 1 week
Sponsore

Rename linux_set_upcall_kse() to linux_set_upcall().

This matches the rename of cpu_set_upcall_kse() in
5c2cf818454375536fda522ba83cf67c50929e6b.

Reviewed by: kib, emaste
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D29295

show more ...

x86: Reduce code duplication in cpu_fork() and cpu_copy_thread().

Add copy_thread() to hold shared code.

Reviewed by: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://revi

x86: Reduce code duplication in cpu_fork() and cpu_copy_thread().

Add copy_thread() to hold shared code.

Reviewed by: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29228

show more ...

x86: Update some stale comments in cpu_fork() and cpu_copy_thread().

Neither of these routines allocate stacks.

Reviewed by: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https

x86: Update some stale comments in cpu_fork() and cpu_copy_thread().

Neither of these routines allocate stacks.

Reviewed by: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29227

show more ...

x86: Always use clean FPU and segment base state for new kthreads.

Reviewed by: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29208

x86: Copy the FPU/XSAVE state from the creating thread to new threads.

POSIX states that new threads created via pthread_create() should
inherit the "floating point environment" from the creating th

x86: Copy the FPU/XSAVE state from the creating thread to new threads.

POSIX states that new threads created via pthread_create() should
inherit the "floating point environment" from the creating thread.

Discussed with: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29204

show more ...

amd64: Only update fsbase/gsbase in pcb for curthread.

Before the pcb is copied to the new thread during cpu_fork() and
cpu_copy_thread(), the kernel re-reads the current register values in
case the

amd64: Only update fsbase/gsbase in pcb for curthread.

Before the pcb is copied to the new thread during cpu_fork() and
cpu_copy_thread(), the kernel re-reads the current register values in
case they are stale. This is done by setting PCB_FULL_IRET in
pcb_flags.

This works fine for user threads, but the creation of kernel processes
and kernel threads do not follow the normal synchronization rules for
pcb_flags. Specifically, new kernel processes are always forked from
thread0, not from curthread, so adjusting pcb_flags via a simple
instruction without the LOCK prefix can race with thread0 running on
another CPU. Similarly, kthread_add() clones from the first thread in
the relevant kernel process, not from curthread. In practice, Netflix
encountered a panic where the pcb_flags in the first kthread of the
KTLS process were trashed due to update_pcb_bases() in
cpu_copy_thread() running from thread0 to create one of the other KTLS
threads racing with the first KTLS kthread calling fpu_kern_thread()
on another CPU. In the panicking case, the write to update pcb_flags
in fpu_kern_thread() was lost triggering an "Unregistered use of FPU
in kernel" panic when the first KTLS kthread later tried to use the
FPU.

Reported by: gallatin
Discussed with: kib
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29023

show more ...

Move SV_ABI_ERRNO translation into linux-specific code, to simplify
the syscall path and declutter it a bit. No functional changes intended.

Reviewed by: kib (earlier version)
MFC after: 2 weeks
Sp

Move SV_ABI_ERRNO translation into linux-specific code, to simplify
the syscall path and declutter it a bit. No functional changes intended.

Reviewed by: kib (earlier version)
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26378

show more ...

Add amd64 procctl(2) ops to manage forced LA48/LA57 VA after exec.

Tested by: pho (LA48 hardware)
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D25273

Retire procfs-based process debugging.

Modern debuggers and process tracers use ptrace() rather than procfs
for debugging. ptrace() has a supserset of functionality available
via procfs and new deb

Retire procfs-based process debugging.

Modern debuggers and process tracers use ptrace() rather than procfs
for debugging. ptrace() has a supserset of functionality available
via procfs and new debugging features are only added to ptrace().
While the two debugging services share some fields in struct proc,
they each use dedicated fields and separate code. This results in
extra complexity to support a feature that hasn't been enabled in the
default install for several years.

PR: 244939 (exp-run)
Reviewed by: kib, mjg (earlier version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D23837

show more ...

amd64: move pcb out of kstack to struct thread.

This saves 320 bytes of the precious stack space.

The only negative aspect of the change I can think of is that the
struct thread increased by 320 by

amd64: move pcb out of kstack to struct thread.

This saves 320 bytes of the precious stack space.

The only negative aspect of the change I can think of is that the
struct thread increased by 320 bytes obviously, and that 320 bytes are
not swapped out anymore. I believe the freed stack space is much more
important than that. Also, current struct thread size is 1392 bytes
on amd64, so UMA will allocate two thread structures per (4KB) slab,
which leaves a space for pcb without increasing zone memory use.

Reviewed by: alc, markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D22138

show more ...

Centralize __pcpu definitions.

Many extern struct pcpu <something>__pcpu declarations were
copied/pasted in sources. The issue is that the definition is MD, but
it cannot be provided by machine/pcp

Centralize __pcpu definitions.

Many extern struct pcpu <something>__pcpu declarations were
copied/pasted in sources. The issue is that the definition is MD, but
it cannot be provided by machine/pcpu.h due to actual struct pcpu
defined in sys/pcpu.h later than the inclusion of machine/pcpu.h.
This forced the copying when other code needed direct access to
__pcpu. There is no way around it, due to machine/pcpu.h supplying
part of struct pcpu fields.

To work around the problem, add a new machine/pcpu_aux.h header, which
should fill any needed MD definitions after struct pcpu definition is
completed. This allows to remove copies of __pcpu spread around the
source. Also on x86 it makes it possible to remove work arounds like
OFFSETOF_CURTHREAD or clang specific warnings supressions.

Reported and tested by: lwhsu, bcran
Reviewed by: imp, markj (previous version)
Discussed with: jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D21418

show more ...