sbin/hammer2: Silence -Waddress-of-packed-member warnings

Not a problem on x86_64, but silence warnings caused by callers
passing address of packed (possibly unaligned) struct fields.

sys/vfs/hammer2: Add initial multi-volumes support for HAMMER2

This commit adds initial multi-volumes support for HAMMER2. Maximum
supported volumes is 64. The feature and implementation is similar

sys/vfs/hammer2: Add initial multi-volumes support for HAMMER2

This commit adds initial multi-volumes support for HAMMER2. Maximum
supported volumes is 64. The feature and implementation is similar to
multi-volumes support in HAMMER1.

1. ondisk changes
=================
This commit bumps volume header version from 1 to 2, and adds four new
volume header fields using reserved fields in version 1. Other ondisk
structures are unchanged.
* "volu_id" - volume id from 0 to 63, where 0 represents root volume.
* "nvolumes" - number of volumes. All volumes have same the same value.
* "total_size" - sum of "volu_size" in volumes. All volumes have the
same value.
* "volu_loff[HAMMER2_MAX_VOLUMES]" - A 512 bytes table which contains
start offset of max 64 volumes within "total_size". All volumes have
the same value.

Version 1 volume header has 0 for above fields, so HAMMER2 internally
treats "nvolumes" as 1, and "total_size" as "volu_size" to be able to
handle version 1 and 2 transparently.

All volumes have 4 headers, but only root volume ones are relevant.
Non-root volume headers have their own unique "volu_id" and "volu_size",
but other fields are unimportant and never used. Non-root volume headers
have sroot blockset[i] whose type is HAMMER2_BREF_TYPE_INVALID. Non-root
volume headers don't have boot/aux area, so freemap area start from
offset 0. Non-root volume headers are readonly and never updated after
creation. This means non-root volumes are just extra storage to extend
fs size and internally make up a single virtual volume whose size is
"total_size".

It currently doesn't automatically upgrade an existing version 1 fs to
version 2. Only newly created fs becomes version 2 for now.

2. volumes layout
=================
Basically similar to HAMMER1. A first block device argument provided for
newfs_hammer2(8) becomes the root volume, and if specified remaining
devices extend "total_size" as non-root volumes. All volumes except for
the last one have 1GiB (freemap level1) aligned "volu_size".

This means each volume's start offset within "total_size" is also 1GiB
(freemap level1) aligned. The start offsets of volumes are stored in
volu_loff[HAMMER2_MAX_VOLUMES]. Each volu_loff[n] (0 <= n < nvolumes)
represents start offset of volume n within "total_size". Unused volumes
have -1 for volu_loff[n].
e.g. If a fs consists of 1 volume, volu_loff[0] has 0 and rests have -1.
e.g. If a fs consists of 3 volumes, x GiB root volume, y GiB volume,
and z GiB volume, volu_loff[0] has 0, volu_loff[1] has x, volu_loff[2]
has x+y, and rests have -1.

Low level I/O function in HAMMER2 uses this linear offsets table to
determine a device vnode to use and relative offset within the device
vnode, for a given blockref's "data_off". This is different from HAMMER1
where logical offset had embedded volume id bits (i.e. there were holes
in logical address space). HAMMER2 needs this table to support multi-
volumes without changing current logical offset mechanism.

Unless all volumes are specified and mountable, mount_hammer2(8) fails
like it failed in HAMMER1. This also applies to other userspace commands
which require volumes specification, except for fstyp(8).

3. userspace commands
=====================
Basically same as or similar to HAMMER1.
* newfs_hammer2(8) takes a list of block device paths as argv[].
* mount_hammer2(8) takes block device paths or names in "a:b:c:..."
format.
* hammer2(8) takes block device paths or names in "a:b:c:..." format for
directives which require volumes specification. This commit also adds
"volume-list" directive and an ioctl command HAMMER2IOC_VOLUME_LIST,
which are similar to the one in HAMMER1.
* fsck_hammer2(8) takes device paths or names in "a:b:c:..." format.
* fstyp(8) takes device paths in "path1:path2:path3:..." format.

4. limitations
==============
* hammer2(8) "info" directive ignores multi-volumes block devices.
* hammer2(8) "growfs" directive doesn't support multi-volumes fs.
* fstyp(8) is unable to find PFS label via -l option if the PFS inode or
its parent indirect blocks are located beyond root volume.
* hammer2(8) doesn't support "volume-add" and "volume-del" directives
which existed in HAMMER1, and there is currently no plan to support.

show more ...

sbin/hammer2: Fix hammer2_pfssubtype_to_str() retval for NONE

cac06d4ec621562239475e76988ab16c661f3430 initially added this
based on cmd_pfs_list() implementation, but change it to return
"NONE" for

sbin/hammer2: Fix hammer2_pfssubtype_to_str() retval for NONE

cac06d4ec621562239475e76988ab16c661f3430 initially added this
based on cmd_pfs_list() implementation, but change it to return
"NONE" for HAMMER2_PFSSUBTYPE_NONE so that this function makes
sense in other contexts.

show more ...

sbin/hammer2: Don't use 1 byte size for 0 radix bits

Same as 893aa36e0111c1418f260bd8d1092b7f22ea010e for userspace code.

sbin/hammer2: "Labels" -> "Label"

sbin/hammer2: Sync "info" output with "pfs-list"

These two retrieve the same info in different ways,
so show the same info instead of "info" just printing PFS label.

sbin/hammer2: Change "info" callback to take const void*

sbin/hammer2: Cleanup "info" with cmd_callback typedef

sys/vfs/hammer2: Remove HAMMER2_MINIO* macros

HAMMER2_MINIO* were added in 1a7cfe5ae3c897f704a358fd3e556a55e430dcb1
in 2013 to switch HAMMER2_MINIO* values with ifdefs when ondisk format
wasn't stab

sys/vfs/hammer2: Remove HAMMER2_MINIO* macros

HAMMER2_MINIO* were added in 1a7cfe5ae3c897f704a358fd3e556a55e430dcb1
in 2013 to switch HAMMER2_MINIO* values with ifdefs when ondisk format
wasn't stable.

This is no longer needed, and using both HAMMER2_MINIO* and HAMMER2_LBUF*
should be avoided. Users of HAMMER2_MINIO* exist in several userspace
programs which walk through blockrefs from top to bottom.

show more ...

sbin/hammer2: Cleanup "info"

Only volume, inode, indirect blockref (out of 9 types) needed for this.

Fix some cases where %zd was used wrongly across the tree.

sbin/hammer2: Add /dev/ad* and /dev/vn* to "info"/"mountall" targets

sbin/hammer2: Test /dev/mampper for "info"/"mountall" targets

Try dirents under /dev/mapper in addition to /dev/serno.
Also ignore ".", ".." for both serno/dm, and "control" for dm.

sbin/hammer2: Use SEEK_SET

sbin/hammer2: Fix remaining type warnings on Linux distros

Bring in local changes existed in https://github.com/kusumi/lh1.
These ondisk fields are not of char*.

sbin/hammer2: Don't test uuid on "info"/"mountall" if uuid is 0

Don't terminate if HAMMER2 device/partition has 0 uuid.

sbin/hammer2: Remove unnecessary { ... } scope

sbin/hammer2: Minor fix

hammer2 - Cleanup 'info' command

* hammer2 info validates the partition info, but assumed that
the filesystem would be a disklabel. Add code to validate
against GPT slices too by checking the f

hammer2 - Cleanup 'info' command

* hammer2 info validates the partition info, but assumed that
the filesystem would be a disklabel. Add code to validate
against GPT slices too by checking the filesystem uuid.

Reported-by: ftigeot

show more ...

sbin/hammer2: Use volatile sig_atomic_t

hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing.

* Add the xxhash. This is a high-speed non-cryptographic hash code
algorithm. Sam pointed me at the site, the code is

hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing.

* Add the xxhash. This is a high-speed non-cryptographic hash code
algorithm. Sam pointed me at the site, the code is available on
github and is BSD licensed:

git://github.com/Cyan4973/xxHash.git

This hash has good distribution and is very fast.

* Change HAMMER2 to default to using xxhash64 instead of iscsi_crc32().
xxhash can process data at several GBytes/sec where as even the
multi-table iscsi_crc32() can only do around 500 MBytes/sec, which
is too slow for today's modern storage subsystems (NVME can nominally
do 1.5-2.5 GBytes/sec, and high-end cards can do 5GBytes/sec).

* There are four major paths that eat tons of CPU in H2:

- The XIO path does a ton of allocation/deallocation and synchronous
messaging. This has not yet been fixed.

- The check code (when it was iscsi_crc32()) slowed everything down.
This is fixed, the default check code is now xxhash64.

- The check code was being called over and over again for the same cached
buffer due to the hammer2_chain_t structure being thrown away.

Currently a hack involving a mask stored in the underlying DIO is being
used to indicate that the check code was previously valid. This is
strictly temporary. The actual mask will have to be stored in the
device buffer cache buffer and a second one in the chain structure.

The chain structure must be made persistent as well (not yet done).

- The DEDUP code was also calling iscsi_crc32() redundantly (at least for
reads).

The read path has been fixed. The write path is doable but requires more
coding (not yet fixed).

- The logical file cluster_read() in the kernel was not doing any read-ahead
due to H2 not implementing BMAP, creating long synchronous latencies.

The kernel code for cluster_read() and cluster_readcb() has been fixed
to do read-ahead whether a logical BMAP is implemented or not. H2 will
now pipeline reads.

Suggested-by: Samuel J. Greear <sjg@thesjg.com> (xxhash)

show more ...

hammer2 - Add 'info' and 'mountall' directives

* hammer2 info - scans all block devices with hammer2 partitions and
displays available super-root entries.

* hammer2 mountall - scans all block dev

hammer2 - Add 'info' and 'mountall' directives

* hammer2 info - scans all block devices with hammer2 partitions and
displays available super-root entries.

* hammer2 mountall - scans all block devices with hammer2 partitions
and mounts their @LOCAL PFS in /var/hammer2/LOCAL.serno.s%d%c
in the background, waiting until the mounts complete or 15 seconds
without a status change.

The idea here is to bring all local block devices online, allowing
all PFS's related to the H2 block devices to become active (mounting
one makes them all available), but still giving the system operator
the ability to idle a block device for maintenance by dismounting
all of its actively-mounted PFSs.

A bit confusing, the feature will be ironed out later.

show more ...