Kernel - Fix numerous procfs/ptrace issues (2)

* The last fix wasn't quite right. Fix it for real.

Kernel - Fix numerous procfs/ptrace issues

* Fix interactions during exec and exit. This should fix truss
(bug 2313).

* Reset p_* fields related to tracing on last procfs close for the
process.

kernel: Replace all usage of MALLOC()/FREE() with kmalloc()/kfree().

kernel - Fix race between procfs / proc sysctls and exec, refactor PHOLD/etc

* During a [v]fork/exec sequence the exec will replace the VM space of the
target process. A concurrent 'ps' operation

kernel - Fix race between procfs / proc sysctls and exec, refactor PHOLD/etc

* During a [v]fork/exec sequence the exec will replace the VM space of the
target process. A concurrent 'ps' operation could access the target
process's vmspace as it was being ripped out, resulting in memory
corruption.

* The P_INEXEC test in procfs was insufficient, the exec code itself must
also wait for procfs's PHOLD() on the process to go away before it can
proceed. This should properly interlock the entire operation.

* Can occur with procfs or non-procfs ps's (via proc sysctls).

* Possibly related to the seg-fault issue we have where the user stack gets
corrupted.

* Also revamp PHOLD()/PRELE() and add PSTALL(), changing all manual while()
loops waiting on p->p_lock to use PSTALL().

These functions now integrate a wakeup request flag into p->p_lock
using atomic ops and no longer tsleep() for 1 tick (or hz ticks, or
whatever). Wakeups are issued proactively.

show more ...

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

kernel - Fix marker in sysctl_kern_proc()

* The marker wasn't being marked as a marker, resulting in a
kernel panic when two or more 'ps' commands are running concurrently
and one blocks.

kernel - Fix ps/thread-exit and other related ps races

* Adjust sysctl_kern_proc()'s kernel thread scanning code to use a marker
instead of depending on td remaining on its proper list. Otherwise

kernel - Fix ps/thread-exit and other related ps races

* Adjust sysctl_kern_proc()'s kernel thread scanning code to use a marker
instead of depending on td remaining on its proper list. Otherwise
blocking conditions can rip td out from under us or move it to another
cpu, potentially resulting in a crash or livelock. Index the scan
backwards to avoid live-locking continuous adds to the list.

* Fix a potential race is the zombie removal code vs a ps, p->p_token was
being released too early.

* Adjust lwkt_exit() to wait for the thread's hold count to drop to zero
so lwkt_hold() works as advertised.

show more ...

kernel - Fix token ordering in sysctl_kern_proc_args(), sysctl_kern_proc_cwd()

* We have to PHOLD() first, then get p->p_token, not the other way around.
Use pfind() instead of pfindn().

* Extrac

kernel - Fix token ordering in sysctl_kern_proc_args(), sysctl_kern_proc_cwd()

* We have to PHOLD() first, then get p->p_token, not the other way around.
Use pfind() instead of pfindn().

* Extract and ref the the ncdir from p->p_fd before calling cache_fullpath()
on it to ensure stability.

show more ...

kernel - Major signal path adjustments to fix races, tsleep race fixes, +more

* Refactor the signal code to properly hold the lp->lwp_token. In
particular the ksignal() and lwp_signotify() paths.

kernel - Major signal path adjustments to fix races, tsleep race fixes, +more

* Refactor the signal code to properly hold the lp->lwp_token. In
particular the ksignal() and lwp_signotify() paths.

* The tsleep() path must also hold lp->lwp_token to properly handle
lp->lwp_stat states and interlocks.

* Refactor the timeout code in tsleep() to ensure that endtsleep() is only
called from the proper context, and fix races between endtsleep() and
lwkt_switch().

* Rename proc->p_flag to proc->p_flags

* Rename lwp->lwp_flag to lwp->lwp_flags

* Add lwp->lwp_mpflags and move flags which require atomic ops (are adjusted
when not the current thread) to the new field.

* Add td->td_mpflags and move flags which require atomic ops (are adjusted
when not the current thread) to the new field.

* Add some freeze testing code to the x86-64 trap code (default disabled).

show more ...

kernel - Remove ad-hoc increment/decrement of vm->vm_sysref

* Remove the ad-hoc increment/decrement of vm->vm_sysref when pulling data
out for a ps. Acquire p->p_token instead.

This is an atte

kernel - Remove ad-hoc increment/decrement of vm->vm_sysref

* Remove the ad-hoc increment/decrement of vm->vm_sysref when pulling data
out for a ps. Acquire p->p_token instead.

This is an attempt to determine whether these adhoc operations are
responsible for causing a race that results in the seg-fault issue we
see on monster.

show more ...

kernel - Hold required token when accessing p_flags, adjust kmem access

* Numerous adjustments to p->p_flag were not being done with p->p_token held.
In particular uiomove().

* Replace P_DEADLKTR

kernel - Hold required token when accessing p_flags, adjust kmem access

* Numerous adjustments to p->p_flag were not being done with p->p_token held.
In particular uiomove().

* Replace P_DEADLKTREAT with LWP_DEADLKTREAT in several places where it had
not been previously converted.

* Allow DMAP access in is_globaldata_space() for x86-64

show more ...

kernel - limit allproc_scan() to snapshot

* Limit the allproc_scan() to (roughly) a snapshot of nprocs processes.
If we don't do this it is possible for the scan to race fork/exec'ing
programs a

kernel - limit allproc_scan() to snapshot

* Limit the allproc_scan() to (roughly) a snapshot of nprocs processes.
If we don't do this it is possible for the scan to race fork/exec'ing
programs and essentially run forever.

show more ...

Revert "kernel: Don't disturb about-to-be reaped zombies."

p_nthreads == 0 just means that the process's threads have all exited,
but not necessarily means it's about to be reaped. We still want it

Revert "kernel: Don't disturb about-to-be reaped zombies."

p_nthreads == 0 just means that the process's threads have all exited,
but not necessarily means it's about to be reaped. We still want it
to show up in the ps or top in case it gets stuck during the exit.

Pointed-out-by: dillon@

This reverts commit 9458c398c0b97d30032a6541f8ab452cd93f5a86.

show more ...

kernel: Don't disturb about-to-be reaped zombies.

Processes in zombproc with p_nthreads == 0 are about to be reaped,
so just leave them in peace.

Dragonfly-bug: <http://bugs.dragonflybsd.org/issue1

kernel: Don't disturb about-to-be reaped zombies.

Processes in zombproc with p_nthreads == 0 are about to be reaped,
so just leave them in peace.

Dragonfly-bug: <http://bugs.dragonflybsd.org/issue1996>

show more ...

LIST_FOREACH_MUTUABLE() was tracking processes not held with PHOLD().
Use a normal LIST_FOREACH() instead because the main iterator is being
protected by PHOLD().

Add missing LWPHOLD/LWPRELE in kinfo code.

Allocate lwkt threads from objcache instead of custom per-cpu cache backed
by zone.

Reviewed-by: dillon@

Allocate struct lwp instances with kmalloc() instead of zalloc() for MP safety.

Migrate allocation of proc structures from zones to kmalloc(). This gives us
MP-safety and does not seem to decrease performance (using make buildworld as
benchmark). Using an objcache seems unnece

Migrate allocation of proc structures from zones to kmalloc(). This gives us
MP-safety and does not seem to decrease performance (using make buildworld as
benchmark). Using an objcache seems unnecessary because there isn't much
pre-initialization code we could move to a constructor and there is no
performance gain.

Debatted-With: dillon@, hsu@, corecode@

show more ...

Move the following entries from kern to security

- kern.ps_showallprocs
- kern.ps_showallthreads
- kern.unprivileged_read_msgbuf
- kern.hardlink_check_uid
- kern.hardlink_check_gid

This is only a c

Move the following entries from kern to security

- kern.ps_showallprocs
- kern.ps_showallthreads
- kern.unprivileged_read_msgbuf
- kern.hardlink_check_uid
- kern.hardlink_check_gid

This is only a cosmetic change helping users to find the right sysctls
more easily. And it could help if we want to add more security related
function (eg MAC framework etc).

While here add missing description for three of them.

show more ...

Add a PHOLD/PRELE sequence around a sysctl_out to fix a race against
the reaper.

Submitted-by: "Nicolas Thery" <nthery@gmail.com>

Fix exporting of lwp data.

Submitted-by: Nicolas Thery <nthery@gmail.com>

1:1 Userland threading stage 2.18/4:

Push lwp use a bit further by making some places lwp aware.
This commit deals with ddb, procfs/ptrace and various consumers of
allproc_scan.

1:1 Userland threading stage 2.15/4:

Drop P_ZOMBIE and make SZOMB a real state again.

1:1 Userland threading stage 2.12/4:

Factor out lwp_stat and move P_STOPPED into p_stat.

Reviewed-by: Thomas E. Spanjaard <tgen@netphreax.net>