kernel - Fix posix_fallocate()

* Our newly minted posix_fallocate() system call was ignoring the
offset field.

* Should fix samba, however any tdb databases that were created are
probably corru

kernel - Fix posix_fallocate()

* Our newly minted posix_fallocate() system call was ignoring the
offset field.

* Should fix samba, however any tdb databases that were created are
probably corrupt so you may need to rm -rf /var/db/samba4, restart
samba, and re-create your smb passwords.

Reported-by: Tim Darby

show more ...

sys/kern: Add fdatasync(2)

Based on the following FreeBSD commits in 2016.
295af703a0d7987c6cf4987e7b7f5f07b3ca1221
1c1cc89580f0fbfabaf6f6c7f0f6440eef0c128e

Add the syscall and also add it to pthre

sys/kern: Add fdatasync(2)

Based on the following FreeBSD commits in 2016.
295af703a0d7987c6cf4987e7b7f5f07b3ca1221
1c1cc89580f0fbfabaf6f6c7f0f6440eef0c128e

Add the syscall and also add it to pthread's cancellation point.
The default behavior is same as fsync(2), which is fine but inefficient.

show more ...

sys/kern: Move vop_stdallocate() within vfs_default.c

vop_allocate stuff are usually placed after vop_markatime,
but looking at vfs_default.c this std function should probably
be the last.

sys/kern: Add posix_fallocate(2)

Based on the initial posix_fallocate(2) implementation in FreeBSD
from d91f88f7f38078ac6dd4b0f0e77470d605d1518e in 2011.

There is nothing special or efficient about

sys/kern: Add posix_fallocate(2)

Based on the initial posix_fallocate(2) implementation in FreeBSD
from d91f88f7f38078ac6dd4b0f0e77470d605d1518e in 2011.

There is nothing special or efficient about the default behavior.
VFS has no knowledge of fs details.

show more ...

kernel - Fix /dev/fd/N and clean up the old dup error-code-driven path

* When opening /dev/fd/N, replicate the file pointer for descriptors
that represent vnodes instead of dup()ing. This ensures

kernel - Fix /dev/fd/N and clean up the old dup error-code-driven path

* When opening /dev/fd/N, replicate the file pointer for descriptors
that represent vnodes instead of dup()ing. This ensures that the seek
offset and other fp-related elements are not shared unexpectedly.

* Refactor the open() path to allow dev_dopen() to replace the
struct file by passing a struct file ** instead of a struct file *.
This removes old error-code-based hacks.

* This fixes the shared seek position that fexecve() was operating with
due to its use of /dev/fd/N for scripts.

Reported-by: aly

show more ...

kernel - Refactor GETATTR_QUICK() -> GETATTR_LITE()

* Refactor GETATTR_QUICK() into GETATTR_LITE() and use struct
vattr_lite instead of struct vattr. The original GETATTR_QUICK()
just used a st

kernel - Refactor GETATTR_QUICK() -> GETATTR_LITE()

* Refactor GETATTR_QUICK() into GETATTR_LITE() and use struct
vattr_lite instead of struct vattr. The original GETATTR_QUICK()
just used a struct vattr.

This change ensures that users of this new VOP do not attempt to
access attr fields that are not populated.

Suggested-by: mjg

show more ...

kernel/vm: Rename *_putpages()'s 'sync' argument to 'flags'.

It used to be a boolean but has now become a general flags argument
(for OBJPC_*).

kernel: Remove <sys/n{amei,lookup}.h> from all files that don't need it.

kernel - Normalize the vx_*() vnode interface

* The vx_*() vnode interface is used for initial allocations, reclaims,
and terminations.

Normalize all use cases to prevent the mixing together of

kernel - Normalize the vx_*() vnode interface

* The vx_*() vnode interface is used for initial allocations, reclaims,
and terminations.

Normalize all use cases to prevent the mixing together of the vx_*()
API and the vn_*() API. For example, vx_lock() should not be paired
with vn_unlock(), and so forth.

* Integrate an update-counter mechanism into the vx_*() API, assert
reasonability.

* Change vfs_cache.c to use an int update counter instead of a long.
The vfs_cache code can't quite use the spin-lock update counter API
yet.

Use proper atomics for load and store.

* Implement VOP_GETATTR_QUICK, meant to be a 'quick' version of
VOP_GETATTR() that only retrieves information related to permissions
and ownership. This will be fast-pathed in a later commit.

* Implement vx_downgrade() to convert an exclusive vx_lock into an
exclusive vn_lock (for vnodes). Adjust all use cases in the
getnewvnode() path.

* Remove unnecessary locks in tmpfs_getattr() and don't use
any in tmpfs_getattr_quick().

* Remove unnecessary locks in hammer2_vop_getattr() and don't use
any in hammer2_vop_getattr_quick()

show more ...

hammer2 - Fix inode & chain limits, improve flush pipeline.

* Reorganize VFS_MODIFYING() to avoid certain deadlock conditions and
adjust hammer2 to unconditionally stall in VFS_MODIFYING() when di

hammer2 - Fix inode & chain limits, improve flush pipeline.

* Reorganize VFS_MODIFYING() to avoid certain deadlock conditions and
adjust hammer2 to unconditionally stall in VFS_MODIFYING() when dirty
limits are exceeded.

Make sure VFS_MODIFYING() is called in all appropriate filesystem-
modifying paths.

This ensures that inode and chain structure allocation limits are
adhered to.

* Fix hammer2's wakeup code for the dirty inode count hystereis. This
fixes a situation where stalls due to excessive dirty inodes were waiting
a full second before resuming operation based on the dirty count
hysteresis.

The hysteresis now works as intended:

(1) Trigger a sync when the dirty count reache 50% N.
(2) Stall the frontend when the dirty count reaches 100% N.
(3) Resume the frontend when the diirty count drops to 66% N.

* Fix trigger_syncer() to guarantee that the syncer will flush the
filesystem ASAP when called. If the filesystem is already in a flush,
it will be flushed again.

Previously if the filesystem was already in a flush it would wait one
second before flushing again, which significantly reduces performance
under conditions where the dirty chain limit or the dirty inode limit is
constantly being hit (e.g. chown -R, etc).

Reported-by: tuxillo

show more ...

kernel - Add trigger_syncer(), VFS_MODIFYING()

* Add trigger_syncer(). This function asynchronously triggers the
syncer vnode in the syncer thread for the mount. It is a NOP if
there is no syn

kernel - Add trigger_syncer(), VFS_MODIFYING()

* Add trigger_syncer(). This function asynchronously triggers the
syncer vnode in the syncer thread for the mount. It is a NOP if
there is no syncer thread or syncer vnode.

Will be used by HAMMER2 to pipeline syncs when heavy filesystem
activity over-extends internal memory structures.

* Add VFS_MODIFYING(). This is a hook into the filesystem that
modifying filesystem ops in the kernel will call prior to locking
any vnodes. It allows the filesystem to moderate the over-allocation
of internal structures. Waiting until after the VOP is called is too
late, so we need kernel support for this. Numerous attempts to hack
moderation code into the H2 VOPs have all failed spectacularly.

In H2, over-allocation can occur because H2 must retain disconnected
inodes related to file creation and deletion until the next sync cycle.

show more ...

kernel - Try to fix delayed mtime updates on SHARED+RW maps again

* Attempt to fix continuing problems with mtime updates related to
modifications made via SHARED+RW mmap()s. As evidenced by buil

kernel - Try to fix delayed mtime updates on SHARED+RW maps again

* Attempt to fix continuing problems with mtime updates related to
modifications made via SHARED+RW mmap()s. As evidenced by builds
under synth sometimes getting confused.

* Don't update vp->v_lastwrite_ts in vop_stdclose(). Instead, update
the field and set a new flag VLASTWRITETS upon mmap(SHARED+PROT_WRITE)
or upon mprotect(PROT_WRITE) on a SHARED mmap.

* Clear the flag on any regular write, utimes, or truncation.

* Adjust various filesystems to update mtime from vp->v_lastwrite_ts
only upon UIO_NOCOPY writes, and only if VLASTWRITETS is set.

tmpfs, ufs, hammer, hammer2 adjusted.

show more ...

kernel - Fix mtime for deferred writes from mmap R+W

* When a program modifies a file via a shared R+W mmap and does not
msync() it, the pages do not get synchronized to the filesystem until
pot

kernel - Fix mtime for deferred writes from mmap R+W

* When a program modifies a file via a shared R+W mmap and does not
msync() it, the pages do not get synchronized to the filesystem until
potentially up to 30 seconds later. Our filesystems then record a
mtime as-of when that synchronization occurs rather than when the
modification was made to the pages.

* We cannot easily determine when the modification was made, but we do
track close()'s on file pointers that were opened writable. So what
we do is record the timestamp as of the last close() of a file pointer
that was opened for writing. Then later on we pages are synchronizded,
we use this timestamp for any mtime update. It isn't perfect, but its
better than it was before.

* This is not a perfect fix. The mtime really needs to be updated at the
time of the close() in this situation as well, but it will fix 'make'
related issues for files that are modified with mmap rather than
write() (aka such as ld.gold appears to do).

Reported-by: zrj

show more ...

kernel/vfs: Sort pathconf cases in the default pathconf.

kernel: Use POSIX constants for pathconf defaults (no functional change).

kernel - Flag cdevpriv use (2)

* Don't throw away fp->f_flag bits set prior to the call to
vop_stdopen().

kernel: Remove unused functions vop_panic() and vfs_std{,un}mount().

vop_panic()'s last usage was removed in 2009 (in bdc5802715465c8)
while vfs_stdmount() and vfs_stdunmount() were inherited from F

kernel: Remove unused functions vop_panic() and vfs_std{,un}mount().

vop_panic()'s last usage was removed in 2009 (in bdc5802715465c8)
while vfs_stdmount() and vfs_stdunmount() were inherited from FreeBSD
but never used at all.

show more ...

kernel - Performance tuning

* Use a shared lock in the exec*() code, open, close, chdir, fchdir,
access, stat, and readlink.

* Adjust nlookup() to allow the last namecache record in a path to be

kernel - Performance tuning

* Use a shared lock in the exec*() code, open, close, chdir, fchdir,
access, stat, and readlink.

* Adjust nlookup() to allow the last namecache record in a path to be
locked shared if it is already resolved, and the caller requests it.

* Remove nearly all global locks from critical dsched paths. Defer
creation of the tdio until an I/O actually occurs (huge savings in
the fork/exit paths).

* Improves fork/exec concurrency on monster of static binaries from
14200/sec to 55000/sec+. For dynamic binaries improve from around
2500/sec to 9000/sec or so (48 cores fork/exec'ing different dynamic
binaries). For the same dynamic binary it's more around 5000/sec or
so.

Lots of issues here including the fact that all dynamic binaries load
many shared resources, even hen the binaries are different programs.
AKA libc.so.X and ld-elf.so.2, as well as /dev/urandom (from libc),
and access numerous common path elements.

Nearly all of these paths are now non-contending. The major remaining
contention is in per-vm_page/PMAP manipulation. This is per-page and
concurrent execs of the same program tend to pipeline so it isn't a
big problem.

show more ...

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Apparently everyone's doing it:
http://svnweb.freebsd.org/base?view=revision&revision=251069

Submitted-by: "Eitan Adler" <lists at eitanadl

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Apparently everyone's doing it:
http://svnweb.freebsd.org/base?view=revision&revision=251069

Submitted-by: "Eitan Adler" <lists at eitanadler.com>

show more ...

Remove advertising clause from all that isn't contrib or userland bin.

By: Eitan Adler <lists@eitanadler.com>

VFS quota: hpfs is now read-only, remove support

kernel - Fix NFS panic when competing clients collide on hardlink

* Fix recursive execlusive lock on vnode.

* For example, if several boxes are nfs booting and have the same writable
/var/log, si

kernel - Fix NFS panic when competing clients collide on hardlink

* Fix recursive execlusive lock on vnode.

* For example, if several boxes are nfs booting and have the same writable
/var/log, simultanious attempts to rotate logs can cause a panic due to
higher layers of the OS's hardlink code not being able to detect
duplicate vnodes which are created by some other client racing the same
operation.

* Adds CNP_NOTVP and cn_notvp flag which vop_compat_nlink() uses to notify
NFS's VOP_LOOKUP function to check for a vnode lookup collision.

-Matt

show more ...

kernel - add cache_unlink(), fix a rename issue.

* Add a rollup function called cache_unlink() to handle namecache
effects when unlinking a file.

* Change namecache semantics a bit. When a namec

kernel - add cache_unlink(), fix a rename issue.

* Add a rollup function called cache_unlink() to handle namecache
effects when unlinking a file.

* Change namecache semantics a bit. When a namecache entry is unlinked
we allow it to be left in the topology as long as the vnode survives,
but marked NCF_DESTROYED so it does not conflict with any new entries
that might be named the same, and does not get returned in lookup results.

* This will solve the issue when renaming a file over an empty directory
(destroying the directory) when one or more processes are chdir'd into
that directory. This would cause the process nchdir stuff to get out
of sync with the retained directory vnode because the same namecache
entry would get reused.

show more ...

kernel: Remove newlines from the panic messages that have one.

panic() itself will add a newline.

VFS accounting: remove unneeded code