kernel - Rejigger mount code to add vfs_flags in struct vfsops

* Rejigger the mount code so we can add a vfs_flags field to vfsops,
which mount_init() has visibility to.

* Allows nullfs to flag t

kernel - Rejigger mount code to add vfs_flags in struct vfsops

* Rejigger the mount code so we can add a vfs_flags field to vfsops,
which mount_init() has visibility to.

* Allows nullfs to flag that its mounts do not need a syncer thread.
Previously nullfs would destroy the syncer thread after the
fact.

* Improves dsynth performance (it does lots of nullfs mounts).

show more ...

kernel - Fix smbfs readdir and umount

* Fix a bug in smbfs's readdir that was causing an empty list and also
a later deadlock.

* Fix a bug in smbfs's umount that was panicing the machine.

kernel - Reduce excessive inode hash table allocations

* Reduce excessive inode hash table allocations in various filesystems
(primarily ufs). Introduce vfs_inodehashsize() to calculate a reasona

kernel - Reduce excessive inode hash table allocations

* Reduce excessive inode hash table allocations in various filesystems
(primarily ufs). Introduce vfs_inodehashsize() to calculate a reasonable
hash table size instead of using 'maxvnodes'.

* The new formula is to generally use maxvnodes / 2 (2 x stacking for the
chained hash table). When maxvnodes is large we use maxvnodes / 4, and
if maxvnodes is very large (> 1M vnodes) we use maxvnodes / 8. This
significantly reduces the amount of kernel memory used when mounting
ufs, ext2fs, hpfs, isofs, msdosfs, nfs, ntfs, and smbfs filesystems.

show more ...

kernel - Rename desiredvnodes to maxvnodes, fix deadlock

* Rename the kernel variable 'desiredvnodes' to 'maxvnodes' to match
the sysctl name (which has always been 'maxvnodes'), and to make the

kernel - Rename desiredvnodes to maxvnodes, fix deadlock

* Rename the kernel variable 'desiredvnodes' to 'maxvnodes' to match
the sysctl name (which has always been 'maxvnodes'), and to make the
code more readable.

* Probable fix to a rare mount/umount deadlock which can occur in two
situations (1) When a large number of mounts and unmounts are running
concurrently, and (2) During a umount -a, shutdown, or reboot.

* Considered minor, normal use cases will not reproduce this bug. Only
synth or poudriere can generate the mount/umount traffic necessary to
reproduce this bug.

* Also fixes a minor kernel memory leak of the mount structure which can
occur when a 'df' or filesystem sync races a umount. Also minor.

Reported-by: marino (mount race)

show more ...

kernel/smbfs: Fix type mismatches for smbfs_pbuf_freecnt.

Mostly used for getpbuf_kva() that takes int*.
Noticed while playing with -flto on LINT64 kernel configuration.

kernel - Improve physio performance

* See http://apollo.backplane.com/DFlyMisc/nvme_sys03.txt

* Hash the pbuf system. This chops down spin-lock collisions
at high transaction rates (>150K IOPS)

kernel - Improve physio performance

* See http://apollo.backplane.com/DFlyMisc/nvme_sys03.txt

* Hash the pbuf system. This chops down spin-lock collisions
at high transaction rates (>150K IOPS) by 1000x.

* Implement a pbuf with pre-allocated kernel memory that we
copy into, avoiding page table manipulations and thus
avoiding system-wide invltlb/invlpg IPIs.

* This increases NVMe IOPS tests with three cards from
150K-200K IOPS to 950K IOPS using physio (random read,
4K blocks, from urandom-filled partition, with many
process threads, from 3 NVMe cards in parallel).

* Further adjustments to the vkernel build.

show more ...

kernel/smbfs: Tabify a few lines.

kernel: Remove some dead code.

While here, also remove an unnecessary NULL check in smbfs (M_WAITOK is used)
and use M_ZERO instead of an explicit bzero().

Submitted-by: Edward O'Callaghan <eocalla

kernel: Remove some dead code.

While here, also remove an unnecessary NULL check in smbfs (M_WAITOK is used)
and use M_ZERO instead of an explicit bzero().

Submitted-by: Edward O'Callaghan <eocallaghan@alterapraxis.com>

show more ...

kernel/smbfs: Remove dead code.

kernel - Rewrite vnode ref-counting code to improve performance

* Rewrite the vnode ref-counting code and modify operation to not
immediately VOP_INACTIVE a vnode when its refs drops to 0. By
d

kernel - Rewrite vnode ref-counting code to improve performance

* Rewrite the vnode ref-counting code and modify operation to not
immediately VOP_INACTIVE a vnode when its refs drops to 0. By
doing so we avoid cycling vnodes through exclusive locks when
temporarily accessing them (such as in a path lookup). Shared
locks can be used throughout.

* Track active/inactive vnodes a bit differently, keep track of
the number of vnodes that are still active but have zero refs,
and rewrite the vnode freeing code to use the new statistics
to deactivate cached vnodes.

show more ...

kernel - remove bounds on buffer cache nbuf count for 64-bit

* Remove arbitrary 1GB buffer cache limitation

* Adjusted numerous 'int' fields to 'long'. Even though nbuf is not
likely to exceed 2

kernel - remove bounds on buffer cache nbuf count for 64-bit

* Remove arbitrary 1GB buffer cache limitation

* Adjusted numerous 'int' fields to 'long'. Even though nbuf is not
likely to exceed 2 billion buffers, byte calculations using the
variable began overflowing so just convert that and various other
variables to long.

* Make sure we don't blow-out the temporary valloc() space in early boot
due to nbufs being too large.

* Unbound 'kern.nbuf' specifications in /boot/loader.conf as well.

show more ...

kernel: Use hashdestroy() to free hash tables allocated with hashinit().

It asserts that the table is empty before kfree()ing it.

kernel/smbfs: The netsmb check only applies if it is not the module build.

The necessary files from netsmb are built into the module. In the kernel
it is two separate options.

This fixes buildkerne

kernel/smbfs: The netsmb check only applies if it is not the module build.

The necessary files from netsmb are built into the module. In the kernel
it is two separate options.

This fixes buildkernel.

Reported-by: Francois Tigeot <ftigeot@wolfpond.org>

show more ...

kernel: Replace all usage of MALLOC()/FREE() with kmalloc()/kfree().

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

kernel: Add missing MODULE_VERSION()s for file systems.

The loader will figure out by itself whether to load a module or not,
depending on whether it's already in the kernel config or not, iif
MODUL

kernel: Add missing MODULE_VERSION()s for file systems.

The loader will figure out by itself whether to load a module or not,
depending on whether it's already in the kernel config or not, iif
MODULE_VERSION() is present.

I.e., if MSDOSFS (that has MODULE_VERSION()) is in the config and
msdos_load="YES" is in /boot/loader.conf, msdos.ko will not be loaded
by the loader at all.

Without MODULE_VERSION() it will lead (in the best case) to whining in
dmesg like for ahci or (in the worst case) to weird behavior, such as
for nullfs:

# mount -a
null: vfsload(null): No such file or directory

Therefore, we definitely want MODULE_VERSION() for all new modules.

This commit is the first in a series to add the missing MODULE_VERSION()s.

I know that ufs is not a module, just included it for completeness' sake.

Reported-by: marino, tuxillo

show more ...

smbfs: use libiconv v.2

Add a flags argument to VOP_FSYNC()

* So filesystem code knows whether it is being called from userland or not.

Use SYSREF to reference count struct vnode. v_usecount is now
v_sysref(.refcnt). v_holdcnt is now v_auxrefs. SYSREF's termination state
(using a negative reference count from -0x40000000+) now pla

Use SYSREF to reference count struct vnode. v_usecount is now
v_sysref(.refcnt). v_holdcnt is now v_auxrefs. SYSREF's termination state
(using a negative reference count from -0x40000000+) now places the vnode in
a VCACHED or VFREE state and deactivates it. The vnode is now assigned a
64 bit unique id via SYSREF.

vhold() (which manipulates v_auxrefs) no longer reactivates a vnode and
is explicitly used only to track references from auxillary structures
and references to prevent premature destruction of the vnode. vdrop()
will now only move a vnode from VCACHED to VFREE on the 1->0 transition
of v_auxrefs if the vnode is in a termination state.

vref() will now panic if used on a vnode in a termination state. vget()
must now be used to explicitly reactivate a vnode. These requirements
existed before but are now explicitly asserted.

vlrureclaim() and allocvnode() should now interact a bit better. In
particular, vlrureclaim() will do a better job of finding vnodes to flush
and transition from VCACHED to VFREE, and allocvnode() will do a better
job finding vnodes to reuse without getting blocked by a flush.

allocvnode now uses a real VX lock to sequence vnodes into VRECLAIMED. All
vnode special state processing now uses a VX lock.

Vnodes are now able to be slowly returned to the memory pool when
kern.maxvnodes is reduced at run time.

Various initialization elements have been moved to CTOR/DTOR and are
no longer in the critical path, improving performance. However, since
SYSREF uses atomic_cmpset_int() (aka cmpxchgl), which reduces performance
somewhat, overall performance tends to be about the same.

show more ...

Rename printf -> kprintf in sys/ and add some defines where necessary
(files which are used in userland, too).

Rename malloc->kmalloc, free->kfree, and realloc->krealloc. Pass 1

VNode sequencing and locking - part 3/4.

VNode aliasing is handled by the namecache (aka nullfs), so there is no
longer a need to have VOP_LOCK, VOP_UNLOCK, or VOP_ISSLOCKED as 'VOP'
functions. Bot

VNode sequencing and locking - part 3/4.

VNode aliasing is handled by the namecache (aka nullfs), so there is no
longer a need to have VOP_LOCK, VOP_UNLOCK, or VOP_ISSLOCKED as 'VOP'
functions. Both NFS and DEADFS have been using standard locking functions
for some time and are no longer special cases. Replace all uses with
native calls to vn_lock, vn_unlock, and vn_islocked.

We can't have these as VOP functions anyhow because of the introduction of
the new SYSLINK transport layer, since vnode locks are primarily used to
protect the local vnode structure itself.

show more ...

MASSIVE reorganization of the device operations vector. Change cdevsw
to dev_ops. dev_ops is a syslink-compatible operations vector structure
similar to the vop_ops structure used by vnodes.

Remov

MASSIVE reorganization of the device operations vector. Change cdevsw
to dev_ops. dev_ops is a syslink-compatible operations vector structure
similar to the vop_ops structure used by vnodes.

Remove a huge number of instances where a thread pointer is still being
passed as an argument to various device ops and other related routines.
The device OPEN and IOCTL calls now take a ucred instead of a thread pointer,
and the CLOSE call no longer takes a thread pointer.

show more ...

Remove several layers in the vnode operations vector init code. Declare
the operations vector directly instead of via a descriptor array. Remove
most of the recalculation code, it stopped being nee

Remove several layers in the vnode operations vector init code. Declare
the operations vector directly instead of via a descriptor array. Remove
most of the recalculation code, it stopped being needed over a year ago.

This work is similar to what FreeBSD now does, but was developed along a
different line. Ultimately our vop_ops will become SYSLINK ops for userland
VFS and clustering support.

show more ...

Remove the thread argument from all mount->vfs_* function vectors,
replacing it with a ucred pointer when applicable. This cleans up a
considerable amount of VFS function code that previously delved

Remove the thread argument from all mount->vfs_* function vectors,
replacing it with a ucred pointer when applicable. This cleans up a
considerable amount of VFS function code that previously delved into
the process structure to get the cred, though some code remains.

Get rid of the compatibility thread argument for hpfs and nwfs. Our
lockmgr calls are now mostly compatible with NetBSD (which doesn't use a
thread argument either).

Get rid of some complex junk in fdesc_statfs() that nobody uses.

Remove the thread argument from dounmount() as well as various other
filesystem specific procedures (quota calls primarily) which no longer
need it due to the lockmgr, VOP, and VFS cleanups. These cleanups also
have the effect of making the VFS code slightly less dependant on the
calling thread's context.

show more ...