kernel/ntfs: Fix lockinit() flags.

kernel: Cleanup <sys/uio.h> issues.

The iovec_free() inline very complicates this header inclusion. The
NULL check is not always seen from <sys/_null.h>. Luckily only three
kernel sources needs

kernel: Cleanup <sys/uio.h> issues.

The iovec_free() inline very complicates this header inclusion. The
NULL check is not always seen from <sys/_null.h>. Luckily only three
kernel sources needs it: kern_subr.c, sys_generic.c and uipc_syscalls.c.
Also just a single dev/drm source makes use of 'struct uio'.
* Include <sys/uio.h> explicitly first in drm_fops.c to avoid kfree()
macro override in drm compat layer.
* Use <sys/_uio.h> where only enums and struct uio is needed, but ensure
that userland will not include it for possible later <sys/user.h> use.
* Stop using <sys/vnode.h> as shortcut for uiomove*() prototypes. The
uiomove*() family functions possibly transfer data across kernel/user
space boundary. This header presence explicitly mark sources as such.
* Prefer to add <sys/uio.h> after <sys/systm.h>, but before <sys/proc.h>
and definitely before <sys/malloc.h> (except for 3 mentioned sources).
This will allow to remove <sys/malloc.h> from <sys/uio.h> later on.
* Adjust <sys/user.h> to use component headers instead of <sys/uio.h>.

While there, use opportunity for a minimal whitespace cleanup.

No functional differences observed in compiler intermediates.

show more ...

world/kernel: Use the rounddown2() macro in various places.

Tested-by: zrj

kernel/ntfs: Fix -Wundef.

kernel - Cleanup gcc warnings

* Cleanup gcc warnings at higher optimization levels. This will allow
us to build kernels -O2 or -O3.

kernel/spinlock: Add a description to struct spinlock.

And add it to spin_init() and SPINLOCK_INITIALIZER().

Submitted-by: dclink (see <http://bugs.dragonflybsd.org/issues/2714>)
OK'd-by: dill

kernel/spinlock: Add a description to struct spinlock.

And add it to spin_init() and SPINLOCK_INITIALIZER().

Submitted-by: dclink (see <http://bugs.dragonflybsd.org/issues/2714>)
OK'd-by: dillon

show more ...

kernel/ntfs - Fix buffer overlapping problems.

- Read only one cluster at a time.
- Do not cache boot block.

Taken-from: FreeBSD's r118035
Dragonfly-bug: <http://bugs.dragonflybsd.org/issues/238

kernel/ntfs - Fix buffer overlapping problems.

- Read only one cluster at a time.
- Do not cache boot block.

Taken-from: FreeBSD's r118035
Dragonfly-bug: <http://bugs.dragonflybsd.org/issues/2388>

(Commit refs #2388 and fixes #1853)

show more ...

kernel/ntfs - Fix a possible problem in bufcache.

- Avoid different block size bread() calls for the
same blkno.
- Fix a bug in the code handling holes which could
cause infinite loops.

Taken-f

kernel/ntfs - Fix a possible problem in bufcache.

- Avoid different block size bread() calls for the
same blkno.
- Fix a bug in the code handling holes which could
cause infinite loops.

Taken-from: FreeBSD's r116620

show more ...

kernel/ntfs - Cleanup.

- Remove NetBSD specific code.
- Removed few whitespaces.

kernel/ntfs - Fix warnings when compiling with NTFS_DEBUG

kernel: Silence gcc47's -Wcast-qual warnings.

kernel - Adjust UFS and HAMMER to use uiomovebp()

* Add uiomovebp(), a version of uiomove() which is aware of a locked bp
representing the to or from buffer and can work-around issues related
to

kernel - Adjust UFS and HAMMER to use uiomovebp()

* Add uiomovebp(), a version of uiomove() which is aware of a locked bp
representing the to or from buffer and can work-around issues related
to VM faults causing recursions and deadlocks on the user buffer.

uiomovebp() does not yet detect or handle deadlocks. Implementing
deadlock handling will require a certain degree of finess related to
the vnode and bp locks and we don't want to have to do it unless we
actually deadlock. TODO.

* Adjust UFS, HAMMER, TMPFS, MSDOSFS, NFS, NTFS to use uiomovebp().

show more ...

ntfs - Add NTFS_DEBUG kernel option.

Remove also CVS IDs from the files that still have it.

ntfs - Fix build with NTFS_DEBUG

kernel: Remove some bogus casts to the own type.

kernel: Remove newlines from the panic messages that have one.

panic() itself will add a newline.

Fix some typos.

kernel: Replace all usage of MALLOC()/FREE() with kmalloc()/kfree().

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

kernel - Greatly improve shared memory fault rate concurrency / shared tokens

This commit rolls up a lot of work to improve postgres database operations
and the system in general. With this changes

kernel - Greatly improve shared memory fault rate concurrency / shared tokens

This commit rolls up a lot of work to improve postgres database operations
and the system in general. With this changes we can pgbench -j 8 -c 40 on
our 48-core opteron monster at 140000+ tps, and the shm vm_fault rate
hits 3.1M pps.

* Implement shared tokens. They work as advertised, with some cavets.

It is acceptable to acquire a shared token while you already hold the same
token exclusively, but you will deadlock if you acquire an exclusive token
while you hold the same token shared.

Currently exclusive tokens are not given priority over shared tokens so
starvation is possible under certain circumstances.

* Create a critical code path in vm_fault() using the new shared token
feature to quickly fault-in pages which already exist in the VM cache.
pmap_object_init_pt() also uses the new feature.

This increases fault-in concurrency by a ridiculously huge amount,
particularly on SHM segments (say when you have a large number of postgres
clients). Scaling for large numbers of clients on large numbers of
cores is significantly improved.

This also increases fault-in concurrency for MAP_SHARED file maps.

* Expand the breadn() and cluster_read() APIs. Implement breadnx() and
cluster_readx() which allows a getblk()'d bp to be passed. If *bpp is not
NULL a bp is being passed in, otherwise the routines call getblk().

* Modify the HAMMER read path to use the new API. Instead of calling
getcacheblk() HAMMER now calls getblk() and checks the B_CACHE flag.
This gives getblk() a chance to regenerate a fully cached buffer from
VM backing store without having to acquire any hammer-related locks,
resulting in even faster operation.

* If kern.ipc.shm_use_phys is set to 2 the VM pages will be pre-allocated.
This can take quite a while for a large map and also lock the machine
up for a few seconds. Defaults to off.

* Reorder the smp_invltlb()/cpu_invltlb() combos in a few places, running
cpu_invltlb() last.

* An invalidation interlock might be needed in pmap_enter() under certain
circumstances, enable the code for now.

* vm_object_backing_scan_callback() was failing to properly check the
validity of a vm_object after acquiring its token. Add the required
check + some debugging.

* Make vm_object_set_writeable_dirty() a bit more cache friendly.

* The vmstats sysctl was scanning every process's vm_map (requiring a
vm_map read lock to do so), which can stall for long periods of time
when the system is paging heavily. Change the mechanic to a LWP flag
which can be tested with minimal locking.

* Have the phys_pager mark the page as dirty too, to make sure nothing
tries to free it.

* Remove the spinlock in pmap_prefault_ok(), since we do not delete page
table pages it shouldn't be needed.

* Add a required cpu_ccfence() in pmap_inval.c. The code generated prior
to this fix was still correct, and this makes sure it stays that way.

* Replace several manual wiring cases with calls to vm_page_wire().

show more ...

libiconv: convert NTFS to libiconv.

Partly-taken-from: FreeBSD

HAMMER / VFS_VGET - Add optional dvp argument to VFS_VGET(). Fix readdirplus

* VGET is used by NFS to acquire a vnode given an inode number. HAMMER
requires additional information to determine t

HAMMER / VFS_VGET - Add optional dvp argument to VFS_VGET(). Fix readdirplus

* VGET is used by NFS to acquire a vnode given an inode number. HAMMER
requires additional information to determine the PFS the inode is being
acquired from.

Add an optional directory vnode argument to the VGET. If non-NULL, HAMMER
will extract the PFS information from this vnode.

* Adjust NFS to pass the dvp to VGET when doing a readdirplus.

Note that the PFS is already encoded in file handles, but readdirplus
acquires the attributes for each directory entry it scans (readdir does
not). This fixes readdirplus for NFS served HAMMER PFS exports.

show more ...

Replace all casts of NULL to something with NULL.

Eliminate some dead initialization.

Found-by: LLVM/Clang Static Analyzer

For kmalloc(), MALLOC() and contigmalloc(), use M_ZERO instead of
explicitly bzero()ing.

Reviewed-by: sephe