writers must hold both sched_lock and the process lock; therefore, readers
need only obtain the process lock.

Change the thread ID (thr_id_t) used for 1:1 threading from being a
pointer to the corresponding struct thread to the thread ID (lwpid_t)
assigned to that thread. The primary reason for this change i

Change the thread ID (thr_id_t) used for 1:1 threading from being a
pointer to the corresponding struct thread to the thread ID (lwpid_t)
assigned to that thread. The primary reason for this change is that
libthr now internally uses the same ID as the debugger and the kernel
when referencing to a kernel thread. This allows us to implement the
support for debugging without additional translations and/or mappings.

To preserve the ABI, the 1:1 threading syscalls, including the umtx
locking API have not been changed to work on a lwpid_t. Instead the
1:1 threading syscalls operate on long and the umtx locking API has
not been changed except for the contested bit. Previously this was
the least significant bit. Now it's the most significant bit. Since
the contested bit should not be tested by userland, this change is
not expected to be visible. Just to be sure, UMTX_CONTESTED has been
removed from <sys/umtx.h>.

Reviewed by: mtm@
ABI preservation tested on: i386, ia64

show more ...

Use the proc lock to sleep on a libthr umtx.

Return EINVAL if the contested bit is not set on the umtx passed to
_umtx_unlock() instead of firing a KASSERT.

Initialize 'blocked' to NULL. I think this was a real problem, but I
am not sure about that. The lack of -Werror and the inline noise hid
this for a while.

Turn a KASSERT back into an EINVAL return value. So, next time someone
comes across it, it will turn into a core dump in userland instead of
a kernel panic. I had also inverted the sense of the test,

Turn a KASSERT back into an EINVAL return value. So, next time someone
comes across it, it will turn into a core dump in userland instead of
a kernel panic. I had also inverted the sense of the test, so

Double pointy hat to: mtm

show more ...

Remove a lock held across casuptr() that snuck in last commit.

Move the decision on whether to unset the contested
bit or not from lock to unlock time.

Suggested by: jhb

Fix umtx locking, for libthr, in the kernel.
1. There was a race condition between a thread unlocking
a umtx and the thread contesting it. If the unlocking
thread won the race it may try to wak

Fix umtx locking, for libthr, in the kernel.
1. There was a race condition between a thread unlocking
a umtx and the thread contesting it. If the unlocking
thread won the race it may try to wakeup a thread that
was not yet in msleep(). The contesting thread would then
go to sleep to await a wakeup that would never come. It's
not possible to close the race by using a lock because
calls to casuptr() may have to fault a page in from swap.
Instead, the race was closed by introducing a flag that
the unlocking thread will set when waking up a thread.
The contesting thread will check for this flag before
going to sleep. For now the flag is kept in td_flags,
but it may be better to use some other member or create
a new one because of the possible performance/contention
issues of having to own sched_lock. Thanks to jhb for
pointing me in the right direction on this one.

2. Once a umtx was contested all future locks and unlocks
were happening in the kernel, regardless of whether it
was contested or not. To prevent this from happening,
when a thread locks a umtx it checks the queue for that
umtx and unsets the contested bit if there are no other
threads waiting on it. Again, this is slightly more
complicated than it needs to be because we can't hold
a lock across casuptr(). So, the thread has to check
the queue again after unseting the bit, and reset the
contested bit if it finds that another thread has put
itself on the queue in the mean time.

3. Remove the if... block for unlocking an uncontested
umtx, and replace it with a KASSERT. The _only_ time
a thread should be unlocking a umtx in the kernel is
if it is contested.

show more ...

I was so happy I found the semi-colon from hell that I didn't
notice another typo in the same line. This typo makes libthr unuseable,
but it's effects where counter-balanced by the extra semicolon, w

I was so happy I found the semi-colon from hell that I didn't
notice another typo in the same line. This typo makes libthr unuseable,
but it's effects where counter-balanced by the extra semicolon, which
made libthr remarkably useable for the past several months.

show more ...

It's unfair how one extraneous semi-colon can cause so much grief.

Use __FBSDID().

- Remove the blocked pointer from the umtx structure.
- Use a hash of umtx queues to queue blocked threads. We hash on pid and the
virtual address of the umtx structure. This eliminates cases w

- Remove the blocked pointer from the umtx structure.
- Use a hash of umtx queues to queue blocked threads. We hash on pid and the
virtual address of the umtx structure. This eliminates cases where we
previously held a lock across a casuptr call.

Reviwed by: jhb (quickly)

show more ...

- Create a new lock, umtx_lock, for use instead of the proc lock for
protecting the umtx queues. We can't use the proc lock because we need
to hold the lock across calls to casuptr, which can

- Create a new lock, umtx_lock, for use instead of the proc lock for
protecting the umtx queues. We can't use the proc lock because we need
to hold the lock across calls to casuptr, which can fault.

Approved by: re

show more ...

- Make casuptr return the old value of the location we're trying to update,
and change the umtx code to expect this.

Reviewed by: jeff

- Add an api for doing smp safe locks in userland.
- umtx_lock() is defined as an inline in umtx.h. It tries to do an
uncontested acquire of a lock which falls back to the _umtx_lock()
system

- Add an api for doing smp safe locks in userland.
- umtx_lock() is defined as an inline in umtx.h. It tries to do an
uncontested acquire of a lock which falls back to the _umtx_lock()
system-call if that fails.
- umtx_unlock() is also an inline which falls back to _umtx_unlock() if the
uncontested unlock fails.
- Locks are keyed off of the thr_id_t of the currently running thread which
is currently just the pointer to the 'struct thread' in kernel.
- _umtx_lock() uses the proc pointer to synchronize access to blocked thread
queues which are stored in the first blocked thread.

show more ...

Reduce reliance on sys/sysproto.h pollution

Add sys/errno.h, sys/malloc.h, sys/queue.h, and vm/uma.h as needed.

sys/sysproto.h currently includes sys/acl.h which currently includes
sys/param.h, sys

Reduce reliance on sys/sysproto.h pollution

Add sys/errno.h, sys/malloc.h, sys/queue.h, and vm/uma.h as needed.

sys/sysproto.h currently includes sys/acl.h which currently includes
sys/param.h, sys/queue.h, and vm/uma.h which in turn bring in
sys/errno.h sys/malloc.h.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44465

show more ...

PP mutexes: lock: Reduce 'umtx_lock' holding before taking the user lock

There is no need to have it for the priority check (that the thread
doesn't have a higher priority than the mutex's ceiling),

PP mutexes: lock: Reduce 'umtx_lock' holding before taking the user lock

There is no need to have it for the priority check (that the thread
doesn't have a higher priority than the mutex's ceiling), and there's
also no need to take it if the thread doesn't have privileges to set its
priority to the mutex's ceiling.

While here, turn 'su' into a 'bool' and compute the internal priority
corresponding to the mutex's ceiling once and for all, putting it in new
'new_pri'.

Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44045

show more ...

PP mutexes: lock: Check if priority is too high against base one

Doing this instead of using the current (user) priority, which includes
current lendings, prevents gratuitous failures for threads in

PP mutexes: lock: Check if priority is too high against base one

Doing this instead of using the current (user) priority, which includes
current lendings, prevents gratuitous failures for threads involved in
multiple locking groups, where each group is defined as the threads that
can lock a particular PP or PI mutex. No deadlock can occur in this
case. Indeed, if a thread holds such a lock A giving it a higher
priority than the ceiling of some other lock B that is PP, and B is
acquired by another thread, effectively the latter may not be able to
run but this situation can only last until the first thread releases A,
which it will do eventually.

Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44044

show more ...

PP mutexes: unlock: Reset inherited prio regardless of privileges

'uq_inherited_pri' contains the current priority inherited from Priority
Protection mutexes. If -1 is passed through 'm_ceilings[1]

PP mutexes: unlock: Reset inherited prio regardless of privileges

'uq_inherited_pri' contains the current priority inherited from Priority
Protection mutexes. If -1 is passed through 'm_ceilings[1]', meaning
that there are no such mutexes held anymore, this must be reflected into
it by setting it to PRI_MAX, regardless of whether the thread has
privilege to set realtime priorities (PRI_MAX is also obviously not
a realtime priority level). By contrast, it shall not be updated and
the computed 'new_inherited_pri' shall stay unused if the thread doesn't
have the ability to set a realtime priority, possibly keeping an older
such priority acquired previously.

Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43984

show more ...

umtx: Check for errors from suword32()

This is in preparation for annotating copyin() and related functions
with __result_use_check.

Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https

umtx: Check for errors from suword32()

This is in preparation for annotating copyin() and related functions
with __result_use_check.

Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D43144

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

umtx: allow to configure minimal timeout (in nanoseconds)

PR: 270785
Reviewed by: markj, mav
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org

umtx: allow to configure minimal timeout (in nanoseconds)

PR: 270785
Reviewed by: markj, mav
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39584

show more ...

kern_umtx.c do_wait(): correct confusing indent

Sponsored by: The FreeBSD Foundation
MFC after: 3 days