hammer2 - Add hammer2 boot support

* Add hammer2 boot loader support. The hammer2 boot loader accesses the
hammer2 mount starting at the super-root. To use hammer2 boot support
you need to hav

hammer2 - Add hammer2 boot support

* Add hammer2 boot loader support. The hammer2 boot loader accesses the
hammer2 mount starting at the super-root. To use hammer2 boot support
you need to have a PFS called 'boot', which will be seen as "/boot" by
the boot loader.

The 'boot' PFS can be created either by creating a boot-specific H2
partition with 'newfs_hammer2 -L boot ...', or by using
'hammer2 pfs-create boot' to create a PFS called 'boot'.

You can access any super-root if desired.

* Since boot is a super-root you can mix boot PFS's and other PFS's on the
same hammer2 filesystem, including creating a boot PFS far later on.

* WARNING!!!!! The 'boot' PFS must be configured without compression.
i.e. 'hammer2 setcomp autozero /boot' (assuming you've mounted it on /boot),
before you install anything. The newfs_hammer2 utility will automatically
handle this if you specify a label name of 'boot' (i.e. -L boot).

* WARNING!!!!! The BIOS may not be able to reliably access very high
sector numbers on large hard drives, so to be safe we recommend creating
a small 'a' partition anyway, unless the drive / main hammer2 filesystem
is small enough to not cause problems.

* Add %qx support to boot2's mini printf().

* Fix ipdata->comp_algo propagation bug in hammer2_inode_create()

* newfs_hammer2 and the pfs_create ioctl will set comp_algo to AUTOZERO
for PFS's called "boot" automatically.

show more ...

hammer2 - Cleanup and bugfix pass to hammer2 setcomp paths

* HAMMER2 will attempt to use LZ4 compression by default now.

* hammer2 setcomp now accepts text labels for the compression type and
acc

hammer2 - Cleanup and bugfix pass to hammer2 setcomp paths

* HAMMER2 will attempt to use LZ4 compression by default now.

* hammer2 setcomp now accepts text labels for the compression type and
accepts a compression level of 0 to set the default, which is simply
passed through to the VFS.

* Specify force and recursion flags in hammer2's main options rather than
after the setcomp command.

* Allow a list of paths to be specified rather than just one.

* The HAMMER2IOC_INODE_SET ioctl now only sets field elements which can
actually be modified by userland i.e. only the comp_algo field. It
used to set everything, including the indirect block table.

* Do not modify the hammer2_inode if the compression method is not being
changed.

* Change the zero-block testing logic a little, with related code cleanups.

show more ...

hammer2 - Adjust newfs_hammer2 for recent media changes

* Adjust newfs_hammer2 for recent media changes. The freemap is now based
in the volume header with a blockref set instead of a single bloc

hammer2 - Adjust newfs_hammer2 for recent media changes

* Adjust newfs_hammer2 for recent media changes. The freemap is now based
in the volume header with a blockref set instead of a single blockref.

show more ...

hammer2 - Start working on the freemap (note: freemap not yet operational)

* hammer2 show now tries to dump all four volume headers

* Minimum storage allocation chunk size is 1K, increase MIN_RADIX

hammer2 - Start working on the freemap (note: freemap not yet operational)

* hammer2 show now tries to dump all four volume headers

* Minimum storage allocation chunk size is 1K, increase MIN_RADIX and
MIN_ALLOC appropriately.

* Preliminary reserved block mappings for the freemap indirect and bitmap
blocks.

* Preliminary freemap design will use normal indirect blocks (but with
a different bref.type name). The check area of the bref will be used to
store additional freemap hints.

The volume header will contain a blockref to the root of the freemap,
and newfs_hammer2 now initializes this blockref.

* Bring major hammer2_disk.h media structure comments up to snuff.

* Formalize blockref.methods and set proper defaults. This field contains
the crc/check and compression methods for a blockref.

show more ...

hammer2 - Messaging layer separation work part 3

* Move more hammer2 vfs message code into kern_dmsg.c, renaming and
implementing callbacks as needed.

* Move hammer2_icrc.c (the iscsi crc support

hammer2 - Messaging layer separation work part 3

* Move more hammer2 vfs message code into kern_dmsg.c, renaming and
implementing callbacks as needed.

* Move hammer2_icrc.c (the iscsi crc support) to libkern/icrc32.c

show more ...

hammer2 - Add peer_type field to LNK_CONN and LNK_SPAN

* Adds a peer_type field allowing connections to identify what they are
(i.e. a HAMMER2 mount, a CLUSTER controller, a BLOCK device controlle

hammer2 - Add peer_type field to LNK_CONN and LNK_SPAN

* Adds a peer_type field allowing connections to identify what they are
(i.e. a HAMMER2 mount, a CLUSTER controller, a BLOCK device controller).

Rename the pfs_type field in the volume header, which never made much
sense, to peer_type, which now does make sense though for the moment
the only value it can have is HAMER2_PEER_HAMMER2.

* Filter HAMMER2_PEER_HAMMER2 peer types by pfs_clid to reduce unnecessary
LNK_SPAN traffic being transmitted to a HAMMER2 mount (in the kernel).

* Minor cleanup.

show more ...

hammer2 - Flesh out span code, API cleanups

* Cleanup the transactional APIs and add a few functions to help with
simple (error code only) message replies.

* Better message protocol layering for

hammer2 - Flesh out span code, API cleanups

* Cleanup the transactional APIs and add a few functions to help with
simple (error code only) message replies.

* Better message protocol layering for both the kernel and userland code.

* Kernel now opens a LNK_CONN transaction which will enable the SPAN
protocol on the link and also serve to install a PFS filter (which is
not yet implemented).

Upon success the kernel then initiates the SPAN.

Basically for the kernel:

send LNK_CONN
wait for streaming reply (transaction remains open on both sides)
send LNK_SPAN

TODO: Receive/track LNK_SPANs, each representing a virtual circuit.

TODO: Track LNK_SPANs that match our PFS.

TODO: Issue higher level protocol transaction messages over these
circuits based on VNOPS, caching, mirroring, etc.
(transactional failures can occur when the LNK_SPAN state
changes, forcing a retry, etc).

* Userland now accepts the LNK_CONN and uses the open transaction to
install tracking structures for those connections participating in
the SPAN protocol.

* Userland now installs tracking structures for received SPAN messages.

* Start fleshing out the userland side of the SPAN relay/transmit code.
This will involve yet more structures to track which SPANs are being
relayed over each connection, so changes can be propagated (not yet
implemented).

For userland the TODO is very large so no point iterating it here.

* Kernel now accepts DBG_SHELL replies (basically debug output messages)
and will kprintf() them. DBG_SHELL commands not yet accepted by the
kernel.

show more ...

hammer2 - last_tid renamed to mirror_tid in volume header

* The volume header needs a discrete field to hold the highest-propagated
mirror_tid value.

hammer2 - Crypto handshake work for message stream

* 'hammer2 debug' can now take a destination host argument (and will
default to localhost). This is the debug shell connection.

* Add 'hammer2

hammer2 - Crypto handshake work for message stream

* 'hammer2 debug' can now take a destination host argument (and will
default to localhost). This is the debug shell connection.

* Add 'hammer2 rsainit' to initialize hammer2's keys /etc/hammer2/rsa.*

* Change the 'hammer2 node' directive to 'hammer2 service'.

* Flesh out the initial public key exchange handshake. Currently the
handshake consists of a symmetric 512 byte write and 512 byte read.
The data is encrypted with our private key and the remote end's public
key.

Currently a very simple verifier has been constructed, but we will
ultimately want to use sha or md5 or something like that for the
verifier.

Since I am doing a double-encryption here the first stage encrypt
has to check that the result does not exceed the modulus (typically
bit 7 of the first byte must be zero). If it does we increment
a 32 bit quantity in our pad*[] area and retry until we get a good
result.

* The exchange is used to calculate the AES session key. Session
encryption is not yet implemented. A random session key is sent
by both ends. The actual session key will be the XOR of the one
we send and the one we receive.

* When a connection is accepted the remote end's public key is looked
up in /etc/hammer2/remotes/<IP>.pub. If this file does not exist
then the connection is not allowed.

show more ...

hammer2 - Add pfs_list, pfs_create, pfs_delete

* Add functions to manage PFSs created under the super-root.

newfs_hammer2 - cleanup, rename redo area to aux area, add default label

* The -L option is no longer mandatory. The label will default to 'ROOT'
if not specified.

* The redo area in the volume

newfs_hammer2 - cleanup, rename redo area to aux area, add default label

* The -L option is no longer mandatory. The label will default to 'ROOT'
if not specified.

* The redo area in the volume header has been renamed to the aux area.

show more ...

hammer2 - initial mkdir skeleton

* Code up VOP_NMKDIR() to test the chain creation code and VOP_READDIR.

* Code up VOP_NLOOKUPDOTDOT.

* Adjust newfs_hammer2 to set alloc_tid to 16, so we can reser

hammer2 - initial mkdir skeleton

* Code up VOP_NMKDIR() to test the chain creation code and VOP_READDIR.

* Code up VOP_NLOOKUPDOTDOT.

* Adjust newfs_hammer2 to set alloc_tid to 16, so we can reserve some
inode numbers for the super-root and root.

* Add another field to the media inode_data for inode allocation within
a PFS.

Note: These are mostly skeletons. We don't flush modified chain elements
back to the disk yet and the chain create path can't create indirect
blocks yet.

show more ...

hammer2 - Adjust chain lookup/iteration API, add readdir

* Adjust the chain lookup/iteration API to take a key range key_beg to key_end
(inclusive), instead of a key/mask pair.

* Add and initiali

hammer2 - Adjust chain lookup/iteration API, add readdir

* Adjust the chain lookup/iteration API to take a key range key_beg to key_end
(inclusive), instead of a key/mask pair.

* Add and initialize a parent inode pointer in hammer2_inode{}.

* Adjust the directory key hash to always OR with 0x8000U in order to
guarantee that some low valued codes remain unused so we can use them
for pseudo directory cookies (for '.' and '..').

newfs2_hammer is required on the test volume with this change.

* Implement hammer2_vop_readdir().

show more ...

hammer2 - correct icrc generation for sector 0

* Correct the icrc generation for sector 0. Other icrc's are also stored
in sector 0 and have be generated first.

* Write the volume header out to a

hammer2 - correct icrc generation for sector 0

* Correct the icrc generation for sector 0. Other icrc's are also stored
in sector 0 and have be generated first.

* Write the volume header out to all volume header alternates. All four
are, in fact, master copies. The mount code will find the best one.

show more ...

hammer2 - Initial newfs_hammer2 implementation

* This is a rough start for the newfs_hammer2 implementation.

* Creates the volume header, super-root inode, and named root inode.