Revision tags: v6.2.1, v6.2.0, v6.3.0, v6.0.1, v6.0.0, v6.0.0rc1, v6.1.0, v5.8.3, v5.8.2, v5.8.1, v5.8.0, v5.9.0, v5.8.0rc1, v5.6.3, v5.6.2, v5.6.1, v5.6.0, v5.6.0rc1, v5.7.0, v5.4.3, v5.4.2, v5.4.1, v5.4.0, v5.5.0, v5.4.0rc1, v5.2.2, v5.2.1, v5.2.0, v5.3.0, v5.2.0rc, v5.0.2, v5.0.1, v5.0.0, v5.0.0rc2, v5.1.0, v5.0.0rc1, v4.8.1, v4.8.0, v4.6.2, v4.9.0, v4.8.0rc, v4.6.1, v4.6.0, v4.6.0rc2, v4.6.0rc, v4.7.0, v4.4.3, v4.4.2, v4.4.1, v4.4.0, v4.5.0, v4.4.0rc, v4.2.4, v4.3.1, v4.2.3, v4.2.1, v4.2.0, v4.0.6, v4.3.0, v4.2.0rc, v4.0.5, v4.0.4, v4.0.3, v4.0.2, v4.0.1, v4.0.0, v4.0.0rc3, v4.0.0rc2, v4.0.0rc, v4.1.0, v3.8.2, v3.8.1, v3.6.3, v3.8.0, v3.8.0rc2, v3.9.0, v3.8.0rc, v3.6.2, v3.6.1, v3.6.0, v3.7.1, v3.6.0rc, v3.7.0 |
#
355d67fc |
| 19-Sep-2013 |
Matthew Dillon <dillon@apollo.backplane.com> |
hammer2 - Merge Daniel Flores's HAMMER2 GSOC project into the main tree
* This merge contains work primarily by Daniel Flores, and also by Matthew Dillon. Daniel's work is focused around adding t
hammer2 - Merge Daniel Flores's HAMMER2 GSOC project into the main tree
* This merge contains work primarily by Daniel Flores, and also by Matthew Dillon. Daniel's work is focused around adding the compression infrastructure while my work was focused around changing HAMMER2's I/O infrastructure to work with the compression code.
Daniel's work involved adding the compression functions and heuristics, modifying mainly vnops and vfsops to use them, adding the new buffer cache write thread, and adding the new hammer2 utility directives and related ioctls.
My work involved changing the H2 I/O infrastructure to always double-buffer (i.e. logical buffers vs device buffers) because device buffers can now wind up being a different size than the related logical buffers. I also had to make changes to the hammer2_mount and hammer2_pfsmount mechanics and other things to prevent deadlocks.
Daniel's Work
* Add the hammer2 setcomp directive which sets the compression mode on a directory or file. If applie to a directory, the compression mode is inherited by any new files or directories created under the directory. Pre-existing subdirectories and files are not affected.
The directive has a recursive option to recurse down and set the mode on everything underneath.
* Add wthread_bioq and related fields to hammer2_mount to support a buffer cache buffer writing thread. This thread is responsible for calculating compression sizes, allocating device buffer blocks, and compressing logical buffers into the device buffers.
* Implement HAMMER2_COMP_AUTOZERO, HAMMER2_COMP_LZ4, and HAMMER2_COMP_ZLIB compression modes. AUTOZERO is the zero-block detection code. LZ4 will do zero-block-detection and LZ4 otherwise, and ZLIB will do zero-block-detection and gzip otherwise.
This work entails a ton of new files imported from the LZ4 and ZLIB projects plus lots of wiring.
The new files had to be cleaned up considerably, as well, since they were originally intended for userland.
* Move synchronous device buffer handling out of hammer2_vop_write() and into the support thread. Numerous procedures were moved out of hammer2_vnops.c and into hammer2_vfsops.c as well.
This greatly simplifies hammer2_vop_write() as well as the truncate and extend code, and improves the critical-path performance for write() (at least until the buffer cache fills up or gets too far behind).
* Implement semi-synchronous decompression callbacks for read I/O and read-ahead I/O.
* Add HAMMER2IOC_INODE_COMP_REC_SET and HAMMER2IOC_INODE_COMP_REC_SET2 ioctls to support the setcomp directive.
Matthew's Work
* The hammer2_inode copies additional fields from the inode data, allowing the inode data to be deallocated after use.
* Due to the way the buffer cache now operates, multiple deletions of the same chain key can occur within the same transaction. Adjust the RBTREE compare code to handle the case.
* Track chain structure use on a pfsmount-by-pfsmount basis for current and future management of the kmalloc pools used by hammer2.
* Rework the way inodes are locked to track chain modifications.
* Rewrite hammer2_chain_delete_duplicate().
* Rewrite hammer2_trans_init() and the flush code primarily to fix deadlocks in the flush synchronization mechanics.
* Interlock very low level chain operations with a spin lock instead of the full-blown chain lock to deal with potential deadlocks and fix a few SMP races.
* For the moment make all logical buffers 64KB. Not efficient for small files and will be changed back at some point but necessary for efficient compression at the moment.
* Implement an asynchronous buffer cache callback feature.
* Use a localized in-hammer2_inode-structure size field for frontend operations, including extend and truncate, to remove confusion against backend flushes. This way the inode data can be modified only during the flush and not before where it might cause confusion with previously staged flushes.
show more ...
|