sys/vfs/hammer2: Use HAMMER2_VOLUME_BYTES for volume header size

Both HAMMER2_PBUFSIZE and HAMMER2_VOLUME_BYTES are 64KiB,
but HAMMER2_VOLUME_BYTES should be used for volume header size
when explici

sys/vfs/hammer2: Use HAMMER2_VOLUME_BYTES for volume header size

Both HAMMER2_PBUFSIZE and HAMMER2_VOLUME_BYTES are 64KiB,
but HAMMER2_VOLUME_BYTES should be used for volume header size
when explicitly getting / reading a volume header block.
It's been mix of these two.

show more ...

sbin/*hammer2: Include <stdint.h> for intmax_t/uintmax_t

intmax_t/uintmax_t belong to <stdint.h>.
These files don't compile on OpenBSD without explicit includes.

sbin/hammer2: Silence -Waddress-of-packed-member warnings

Not a problem on x86_64, but silence warnings caused by callers
passing address of packed (possibly unaligned) struct fields.

sys/vfs/hammer2: Add initial multi-volumes support for HAMMER2

This commit adds initial multi-volumes support for HAMMER2. Maximum
supported volumes is 64. The feature and implementation is similar

sys/vfs/hammer2: Add initial multi-volumes support for HAMMER2

This commit adds initial multi-volumes support for HAMMER2. Maximum
supported volumes is 64. The feature and implementation is similar to
multi-volumes support in HAMMER1.

1. ondisk changes
=================
This commit bumps volume header version from 1 to 2, and adds four new
volume header fields using reserved fields in version 1. Other ondisk
structures are unchanged.
* "volu_id" - volume id from 0 to 63, where 0 represents root volume.
* "nvolumes" - number of volumes. All volumes have same the same value.
* "total_size" - sum of "volu_size" in volumes. All volumes have the
same value.
* "volu_loff[HAMMER2_MAX_VOLUMES]" - A 512 bytes table which contains
start offset of max 64 volumes within "total_size". All volumes have
the same value.

Version 1 volume header has 0 for above fields, so HAMMER2 internally
treats "nvolumes" as 1, and "total_size" as "volu_size" to be able to
handle version 1 and 2 transparently.

All volumes have 4 headers, but only root volume ones are relevant.
Non-root volume headers have their own unique "volu_id" and "volu_size",
but other fields are unimportant and never used. Non-root volume headers
have sroot blockset[i] whose type is HAMMER2_BREF_TYPE_INVALID. Non-root
volume headers don't have boot/aux area, so freemap area start from
offset 0. Non-root volume headers are readonly and never updated after
creation. This means non-root volumes are just extra storage to extend
fs size and internally make up a single virtual volume whose size is
"total_size".

It currently doesn't automatically upgrade an existing version 1 fs to
version 2. Only newly created fs becomes version 2 for now.

2. volumes layout
=================
Basically similar to HAMMER1. A first block device argument provided for
newfs_hammer2(8) becomes the root volume, and if specified remaining
devices extend "total_size" as non-root volumes. All volumes except for
the last one have 1GiB (freemap level1) aligned "volu_size".

This means each volume's start offset within "total_size" is also 1GiB
(freemap level1) aligned. The start offsets of volumes are stored in
volu_loff[HAMMER2_MAX_VOLUMES]. Each volu_loff[n] (0 <= n < nvolumes)
represents start offset of volume n within "total_size". Unused volumes
have -1 for volu_loff[n].
e.g. If a fs consists of 1 volume, volu_loff[0] has 0 and rests have -1.
e.g. If a fs consists of 3 volumes, x GiB root volume, y GiB volume,
and z GiB volume, volu_loff[0] has 0, volu_loff[1] has x, volu_loff[2]
has x+y, and rests have -1.

Low level I/O function in HAMMER2 uses this linear offsets table to
determine a device vnode to use and relative offset within the device
vnode, for a given blockref's "data_off". This is different from HAMMER1
where logical offset had embedded volume id bits (i.e. there were holes
in logical address space). HAMMER2 needs this table to support multi-
volumes without changing current logical offset mechanism.

Unless all volumes are specified and mountable, mount_hammer2(8) fails
like it failed in HAMMER1. This also applies to other userspace commands
which require volumes specification, except for fstyp(8).

3. userspace commands
=====================
Basically same as or similar to HAMMER1.
* newfs_hammer2(8) takes a list of block device paths as argv[].
* mount_hammer2(8) takes block device paths or names in "a:b:c:..."
format.
* hammer2(8) takes block device paths or names in "a:b:c:..." format for
directives which require volumes specification. This commit also adds
"volume-list" directive and an ioctl command HAMMER2IOC_VOLUME_LIST,
which are similar to the one in HAMMER1.
* fsck_hammer2(8) takes device paths or names in "a:b:c:..." format.
* fstyp(8) takes device paths in "path1:path2:path3:..." format.

4. limitations
==============
* hammer2(8) "info" directive ignores multi-volumes block devices.
* hammer2(8) "growfs" directive doesn't support multi-volumes fs.
* fstyp(8) is unable to find PFS label via -l option if the PFS inode or
its parent indirect blocks are located beyond root volume.
* hammer2(8) doesn't support "volume-add" and "volume-del" directives
which existed in HAMMER1, and there is currently no plan to support.

show more ...

sbin/fsck_hammer2: Don't print "exceeds volume size" in find_best_zone()

No need to print while seeking for a best zone.

sbin/hammer2: Fix hammer2_pfssubtype_to_str() retval for NONE

cac06d4ec621562239475e76988ab16c661f3430 initially added this
based on cmd_pfs_list() implementation, but change it to return
"NONE" for

sbin/hammer2: Fix hammer2_pfssubtype_to_str() retval for NONE

cac06d4ec621562239475e76988ab16c661f3430 initially added this
based on cmd_pfs_list() implementation, but change it to return
"NONE" for HAMMER2_PFSSUBTYPE_NONE so that this function makes
sense in other contexts.

show more ...

sbin/hammer2: Change hammer2_uuid_to_str() to take const pointer

sbin/hammer2: Add hammer2_pfssubtype_to_str()

sbin/fsck_hammer2: Add -P option to print PFS information

This prints PFS info, in the same format as "pfs-list" in hammer2(8).

The problem with "pfs-list" is it requires HAMMER2 inode, but in orde

sbin/fsck_hammer2: Add -P option to print PFS information

This prints PFS info, in the same format as "pfs-list" in hammer2(8).

The problem with "pfs-list" is it requires HAMMER2 inode, but in order
to mount HAMMER2, PFS label is required. This option provides "pfs-list"
equivalent via block device.

show more ...

fsck_hammer2(8): de-inline couple of functions

Fix -Werror=inline ("call is unlikely and code size would grow") exposed
when building world with -Os.

sys/vfs/hammer2: Remove HAMMER2_MINIO* macros

HAMMER2_MINIO* were added in 1a7cfe5ae3c897f704a358fd3e556a55e430dcb1
in 2013 to switch HAMMER2_MINIO* values with ifdefs when ondisk format
wasn't stab

sys/vfs/hammer2: Remove HAMMER2_MINIO* macros

HAMMER2_MINIO* were added in 1a7cfe5ae3c897f704a358fd3e556a55e430dcb1
in 2013 to switch HAMMER2_MINIO* values with ifdefs when ondisk format
wasn't stable.

This is no longer needed, and using both HAMMER2_MINIO* and HAMMER2_LBUF*
should be avoided. Users of HAMMER2_MINIO* exist in several userspace
programs which walk through blockrefs from top to bottom.

show more ...

sbin/fsck_hammer2: Fix incorrect pfs_type test in ondisk inode

"ipdata.meta.pfs_type & HAMMER2_PFSTYPE_SUPROOT" happened to have
the same result (except HAMMER2_PFSTYPE_DUMMY could also match).

sbin/fsck_hammer2: Add HAMMER2_USE_OPENSSL

sbin/fsck_hammer2: Use RB_GENERATE() for better userspace portability

sbin/fsck_hammer2: Fix siz in tsnprintf()

Prevent small terminal width causing newline behavior in
print_blockref_stats() in verify_blockref().

sbin/fsck_hammer2: Refactor print_blockref_verbose()

sbin/fsck_hammer2: Support regular file

sbin/fsck_hammer2: Adjust indentations for error messages

sbin/fsck_hammer2: Consider device size with less than 4 zones

Unlike HAMMER1 which had volume size requirement, HAMMER2 allows
creation of small filesystem without having 4 zones for volume headers

sbin/fsck_hammer2: Consider device size with less than 4 zones

Unlike HAMMER1 which had volume size requirement, HAMMER2 allows
creation of small filesystem without having 4 zones for volume headers.
Make fsck_hammer2 also consider that, otherwise segfaults if the
device has less than 4 zones.

show more ...

sbin/fsck_hammer2: Make debug mode more verbose on failure

Make failed bref visually clear within radix tree if DebugOpt>1.

sbin/fsck_hammer2: Add debug mode for DebugOpt>1

sbin/fsck_hammer2: Make debug mode more verbose

sbin/fsck_hammer2: Fix delta stats bug (-c option)

It's possible that a cached delta is a part of another delta(s) to be
cached. When this happens, the parent delta needs to include stats
from the c

sbin/fsck_hammer2: Fix delta stats bug (-c option)

It's possible that a cached delta is a part of another delta(s) to be
cached. When this happens, the parent delta needs to include stats
from the child delta, otherwise remaining zone scanning results could
be incorrect.

show more ...

sbin/fsck_hammer2: Remove redundant hammer2_blockref_t in delta stats

sbin/fsck_hammer2: Add -c to cache subtree blockref stats for speed

If blockref scan succeeds and subtree blockref count exceeds a -c
value, cache subtree blockref stats and blockref itself.

This g

sbin/fsck_hammer2: Add -c to cache subtree blockref stats for speed

If blockref scan succeeds and subtree blockref count exceeds a -c
value, cache subtree blockref stats and blockref itself.

This greatly improves speed when multiple PFSes share large parts
of blockrefs via snapshots. If blockrefs were somehow corrupted in
a weird way that exact same copies of blockref appear more than once,
this doesn't work, but should generally be useful.

Note that using a too small -c value may eat memory without much
performance improvement. Also note that using a too large -c value
saves memory but blockrefs may not be cached or used.

A suitable -c value depends on blockref tree structure since it's
a threshold for blockref count. A fs with more than a few GB of
consumption, should use a few thousand or more.

show more ...