kernel: Stop using <stdint.h> during kernel build.

This includes VKERNEL too. Now it should be possible to fully build
kernel with headers only from src/sys/ (except VKERNEL that is using
userla

kernel: Stop using <stdint.h> during kernel build.

This includes VKERNEL too. Now it should be possible to fully build
kernel with headers only from src/sys/ (except VKERNEL that is using
userland headers for obvious reasons, prominently the <signal.h>).

Now <stdint.h> separation from <sys/types.h> could begin.

show more ...

kernel/hammer2: Make C_HASHTABLE static.

Only used in lz4.c locally. Include hammer2.h first to include common
headers first.

hammer2 - Merge Daniel Flores's HAMMER2 GSOC project into the main tree

* This merge contains work primarily by Daniel Flores, and also by
Matthew Dillon. Daniel's work is focused around adding t

hammer2 - Merge Daniel Flores's HAMMER2 GSOC project into the main tree

* This merge contains work primarily by Daniel Flores, and also by
Matthew Dillon. Daniel's work is focused around adding the compression
infrastructure while my work was focused around changing HAMMER2's
I/O infrastructure to work with the compression code.

Daniel's work involved adding the compression functions and heuristics,
modifying mainly vnops and vfsops to use them, adding the new buffer
cache write thread, and adding the new hammer2 utility directives and
related ioctls.

My work involved changing the H2 I/O infrastructure to always double-buffer
(i.e. logical buffers vs device buffers) because device buffers can now
wind up being a different size than the related logical buffers. I also
had to make changes to the hammer2_mount and hammer2_pfsmount mechanics
and other things to prevent deadlocks.

Daniel's Work

* Add the hammer2 setcomp directive which sets the compression mode on a
directory or file. If applie to a directory, the compression mode is
inherited by any new files or directories created under the directory.
Pre-existing subdirectories and files are not affected.

The directive has a recursive option to recurse down and set the mode on
everything underneath.

* Add wthread_bioq and related fields to hammer2_mount to support a
buffer cache buffer writing thread. This thread is responsible for
calculating compression sizes, allocating device buffer blocks, and
compressing logical buffers into the device buffers.

* Implement HAMMER2_COMP_AUTOZERO, HAMMER2_COMP_LZ4, and HAMMER2_COMP_ZLIB
compression modes. AUTOZERO is the zero-block detection code. LZ4 will
do zero-block-detection and LZ4 otherwise, and ZLIB will do
zero-block-detection and gzip otherwise.

This work entails a ton of new files imported from the LZ4 and ZLIB
projects plus lots of wiring.

The new files had to be cleaned up considerably, as well, since they
were originally intended for userland.

* Move synchronous device buffer handling out of hammer2_vop_write() and
into the support thread. Numerous procedures were moved out of
hammer2_vnops.c and into hammer2_vfsops.c as well.

This greatly simplifies hammer2_vop_write() as well as the truncate and
extend code, and improves the critical-path performance for write()
(at least until the buffer cache fills up or gets too far behind).

* Implement semi-synchronous decompression callbacks for read I/O and
read-ahead I/O.

* Add HAMMER2IOC_INODE_COMP_REC_SET and HAMMER2IOC_INODE_COMP_REC_SET2
ioctls to support the setcomp directive.

Matthew's Work

* The hammer2_inode copies additional fields from the inode data, allowing
the inode data to be deallocated after use.

* Due to the way the buffer cache now operates, multiple deletions of the
same chain key can occur within the same transaction. Adjust the RBTREE
compare code to handle the case.

* Track chain structure use on a pfsmount-by-pfsmount basis for current
and future management of the kmalloc pools used by hammer2.

* Rework the way inodes are locked to track chain modifications.

* Rewrite hammer2_chain_delete_duplicate().

* Rewrite hammer2_trans_init() and the flush code primarily to fix
deadlocks in the flush synchronization mechanics.

* Interlock very low level chain operations with a spin lock instead
of the full-blown chain lock to deal with potential deadlocks and
fix a few SMP races.

* For the moment make all logical buffers 64KB. Not efficient for small
files and will be changed back at some point but necessary for efficient
compression at the moment.

* Implement an asynchronous buffer cache callback feature.

* Use a localized in-hammer2_inode-structure size field for frontend
operations, including extend and truncate, to remove confusion against
backend flushes. This way the inode data can be modified only during
the flush and not before where it might cause confusion with previously
staged flushes.

show more ...