Fix various grammatical issues in our messages, documentation etc.

<sys/sysref.h>: Switch to lighter <sys/_malloc.h> header.

* Make <sys/sysref2.h> a kernel only header.
* Remove sys/types.h includes that follow <sys/param.h> in devfs(5).
* Add sys/malloc.h incl

<sys/sysref.h>: Switch to lighter <sys/_malloc.h> header.

* Make <sys/sysref2.h> a kernel only header.
* Remove sys/types.h includes that follow <sys/param.h> in devfs(5).
* Add sys/malloc.h includes where it is actually used in sources.

While there, minor whitespace cleanup.

show more ...

kernel - Remove duplicate TRIM, only trim with the 'trim' mount opt

* ffs_blkfree_cg() was improperly issuing a synchronous
VOP_FREEBLKS() on the underlying device. This issues a
BUF_CMD_FREEBL

kernel - Remove duplicate TRIM, only trim with the 'trim' mount opt

* ffs_blkfree_cg() was improperly issuing a synchronous
VOP_FREEBLKS() on the underlying device. This issues a
BUF_CMD_FREEBLKS stategy op to the underlying device,
which is executed unconditionally. This basically runs
an unconditional TRIM whether the 'trim' mount flag is
specified or not.

Remove the VOP_FREEBLKS() call.

* For softupdates operation, ffs_blkfree() handles the
'trim' mount flag by issuing a BUF_CMD_FREEBLKS and
sequencing the call to ffs_blkfree_cg() when it completes.

When 'trim' was enabled, *two* TRIM operations were being
executed on the underlying device, and prior to our fix,
if 'trim' was not enabled, *one* TRIM operation would
still be executed instead of zero.

* In many situations... possibly even most situations,
trim operations seriously reduce performance due to
being serialized by AHCI or by the target device. It
is not as useful as people often think it should be
on normal filesystems.

* The removal of the unconditional TRIM significantly improves
UFS performance, meaning primarily installkernel's
since DragonFly doesn't use UFS for its main filesystem by
default any more.

* The 'trim' mount option for UFS will still work as advertised
when coupled with softupdates.

show more ...

ufs: Rename a number of UFS constants throughout the tree.

ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS1_MAXSYM

ufs: Rename a number of UFS constants throughout the tree.

ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS1_MAXSYMLINKLEN

This helps to better sepate them from identically named EXT2 constants
which I'll rename in a later commit.

It also helps with my makefs(8) port.

I went with FreeBSD's names, so it is UFS1_MAXSYMLINKLEN even though
we'll probably never have UFS2, but since they are in public headers,
some stuff in dports might benefit from it in the future.

For safety, bump <sys/param.h> too.

show more ...

kernel - Remove repurposebuf

* Remove the repurposebuf hack to prepare for the buffer cache
KVABIO API, which is a better solution.

kernel - Refactor buffer cache code in preparation for vm_page repurposing

* Keep buffer_map but no longer use vm_map_findspace/vm_map_delete to manage
buffer sizes. Instead, reserve MAXBSIZE of

kernel - Refactor buffer cache code in preparation for vm_page repurposing

* Keep buffer_map but no longer use vm_map_findspace/vm_map_delete to manage
buffer sizes. Instead, reserve MAXBSIZE of unallocated KVM for each buffer.

* Refactor the buffer cache management code. bufspace exhaustion now has
hysteresis, bufcount works just about the same.

* Start work on the repurposing code (currently disabled).

show more ...

kernel - Fix incorrect assertion in ffs_reallocblks()

* When softupdates is not enabled reallocblks should not try to free
an unallocated block. This appears to be possible if the reallocation

kernel - Fix incorrect assertion in ffs_reallocblks()

* When softupdates is not enabled reallocblks should not try to free
an unallocated block. This appears to be possible if the reallocation
range covers part of a file with a hole in it. FSX appears to reveal
this case.

* It is also possible that this case is due to some interaction between
the DragonFly buffer cache and UFS that breaks b_offset assignment
assumptions made by UFS, but I couldn't find any particular vector
for the possibility.

show more ...

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Apparently everyone's doing it:
http://svnweb.freebsd.org/base?view=revision&revision=251069

Submitted-by: "Eitan Adler" <lists at eitanadl

Correct BSD License clause numbering from 1-2-4 to 1-2-3.

Apparently everyone's doing it:
http://svnweb.freebsd.org/base?view=revision&revision=251069

Submitted-by: "Eitan Adler" <lists at eitanadler.com>

show more ...

Remove advertising clause from all that isn't contrib or userland bin.

By: Eitan Adler <lists@eitanadler.com>

kernel - Make UFS ihash table per-mount

* Make the UFS ihash table per-mount.

* Scale down the size of the hash table a bit so we have ~4 inodes per
bucket instead of ~1. Works fine for a single

kernel - Make UFS ihash table per-mount

* Make the UFS ihash table per-mount.

* Scale down the size of the hash table a bit so we have ~4 inodes per
bucket instead of ~1. Works fine for a single mount and this way
multiple UFS mounts don't make [as] bloated kmalloc calls.

Submitted-by: vsrinivas

show more ...

kernel: Use NULL for pointers.

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

kernel - Greatly improve shared memory fault rate concurrency / shared tokens

This commit rolls up a lot of work to improve postgres database operations
and the system in general. With this changes

kernel - Greatly improve shared memory fault rate concurrency / shared tokens

This commit rolls up a lot of work to improve postgres database operations
and the system in general. With this changes we can pgbench -j 8 -c 40 on
our 48-core opteron monster at 140000+ tps, and the shm vm_fault rate
hits 3.1M pps.

* Implement shared tokens. They work as advertised, with some cavets.

It is acceptable to acquire a shared token while you already hold the same
token exclusively, but you will deadlock if you acquire an exclusive token
while you hold the same token shared.

Currently exclusive tokens are not given priority over shared tokens so
starvation is possible under certain circumstances.

* Create a critical code path in vm_fault() using the new shared token
feature to quickly fault-in pages which already exist in the VM cache.
pmap_object_init_pt() also uses the new feature.

This increases fault-in concurrency by a ridiculously huge amount,
particularly on SHM segments (say when you have a large number of postgres
clients). Scaling for large numbers of clients on large numbers of
cores is significantly improved.

This also increases fault-in concurrency for MAP_SHARED file maps.

* Expand the breadn() and cluster_read() APIs. Implement breadnx() and
cluster_readx() which allows a getblk()'d bp to be passed. If *bpp is not
NULL a bp is being passed in, otherwise the routines call getblk().

* Modify the HAMMER read path to use the new API. Instead of calling
getcacheblk() HAMMER now calls getblk() and checks the B_CACHE flag.
This gives getblk() a chance to regenerate a fully cached buffer from
VM backing store without having to acquire any hammer-related locks,
resulting in even faster operation.

* If kern.ipc.shm_use_phys is set to 2 the VM pages will be pre-allocated.
This can take quite a while for a large map and also lock the machine
up for a few seconds. Defaults to off.

* Reorder the smp_invltlb()/cpu_invltlb() combos in a few places, running
cpu_invltlb() last.

* An invalidation interlock might be needed in pmap_enter() under certain
circumstances, enable the code for now.

* vm_object_backing_scan_callback() was failing to properly check the
validity of a vm_object after acquiring its token. Add the required
check + some debugging.

* Make vm_object_set_writeable_dirty() a bit more cache friendly.

* The vmstats sysctl was scanning every process's vm_map (requiring a
vm_map read lock to do so), which can stall for long periods of time
when the system is paging heavily. Change the mechanic to a LWP flag
which can be tested with minimal locking.

* Have the phys_pager mark the page as dirty too, to make sure nothing
tries to free it.

* Remove the spinlock in pmap_prefault_ok(), since we do not delete page
table pages it shouldn't be needed.

* Add a required cpu_ccfence() in pmap_inval.c. The code generated prior
to this fix was still correct, and this makes sure it stays that way.

* Replace several manual wiring cases with calls to vm_page_wire().

show more ...

kernel/ufs: Fix mount_point access in ffs_blkfree() when using softdep.

Was going through inode->vnode->v_mount, but vnode was bad so
v_mount was NULL reference.

New way to get mount point: inode->

kernel/ufs: Fix mount_point access in ffs_blkfree() when using softdep.

Was going through inode->vnode->v_mount, but vnode was bad so
v_mount was NULL reference.

New way to get mount point: inode->i_devvp->v_mount.

Reported-by: Thomas Nikolajsen
Dragonfly-bug: <http://bugs.dragonflybsd.org/issue2142>
Submitted-by: Tim Bisson <bissont@mac.com>

show more ...

TRIM support

Signed-off-by: Samuel J. Greear <sjg@thesjg.com>

HAMMER / VFS_VGET - Add optional dvp argument to VFS_VGET(). Fix readdirplus

* VGET is used by NFS to acquire a vnode given an inode number. HAMMER
requires additional information to determine t

HAMMER / VFS_VGET - Add optional dvp argument to VFS_VGET(). Fix readdirplus

* VGET is used by NFS to acquire a vnode given an inode number. HAMMER
requires additional information to determine the PFS the inode is being
acquired from.

Add an optional directory vnode argument to the VGET. If non-NULL, HAMMER
will extract the PFS information from this vnode.

* Adjust NFS to pass the dvp to VGET when doing a readdirplus.

Note that the PFS is already encoded in file handles, but readdirplus
acquires the attributes for each directory entry it scans (readdir does
not). This fixes readdirplus for NFS served HAMMER PFS exports.

show more ...

Use %j to print an ino_t.

Rename printf -> kprintf in sys/ and add some defines where necessary
(files which are used in userland, too).

Remove inode free race warning messages. These were originally added to
determine how often an inode free race occured and are no longer needed.
The actual handling of an inode free race is still in

Remove inode free race warning messages. These were originally added to
determine how often an inode free race occured and are no longer needed.
The actual handling of an inode free race is still in place.

show more ...

Rename functions to avoid conflicts with libc.

Rename functions to avoid conflicts with libc.

Remove FFS function hooks used by UFS. Simply make direct calls from ufs
to ffs. The original ufs routines don't exist anymore anyhow and EXT2 no
longer references UFS files directly. UFS and FFS

Remove FFS function hooks used by UFS. Simply make direct calls from ufs
to ffs. The original ufs routines don't exist anymore anyhow and EXT2 no
longer references UFS files directly. UFS and FFS have been 'one' filesystem
for two decades. These hooks are no longer needed.

show more ...

Replace the the buffer cache's B_READ, B_WRITE, B_FORMAT, and B_FREEBUF
b_flags with a separate b_cmd field. Use b_cmd to test for I/O completion
as well (getting rid of B_DONE in the process). Thi

Replace the the buffer cache's B_READ, B_WRITE, B_FORMAT, and B_FREEBUF
b_flags with a separate b_cmd field. Use b_cmd to test for I/O completion
as well (getting rid of B_DONE in the process). This further simplifies
the setup required to issue a buffer cache I/O.

Remove a redundant header file, bus/isa/i386/isa_dma.h and merge any
discrepancies into bus/isa/isavar.h.

Give ISADMA_READ/WRITE/RAW their own independant flag definitions instead of
trying to overload them on top of B_READ, B_WRITE, and B_RAW. Add a
routine isa_dmabp() which takes a struct buf pointer and returns the ISA
dma flags associated with the operation.

Remove the 'clear_modify' argument to vfs_busy_pages(). Instead,
vfs_busy_pages() asserts that the buffer's b_cmd is valid and then uses
it to determine the action it must take.

show more ...

Get rid of the remaining buffer background bitmap code. It's been turned
off for a while, and it represents a fairly severe hack to the buffer
cache code that just complicates further development.

A number of structures related to UFS and QUOTAS have changed name.

dinode -> ufs1_dinode
dqblk -> ufs_dqblk (and other quota related structures)

In addition, a large number of UFS related structur

A number of structures related to UFS and QUOTAS have changed name.

dinode -> ufs1_dinode
dqblk -> ufs_dqblk (and other quota related structures)

In addition, a large number of UFS related structures and procedures have
been prefixed with 'ufs_' to allow us to split off EXT2FS.

ufs_daddr_t has been moved out of sys/types.h and into vfs/ufs/dinode.h.

The #ifndef header file checks for UFS have been normalized.

show more ...