sbin/hammer2: Make comp|checkmodestr() available in subs.c

These functions have been static in sbin/hammer2/cmd_stat.c,
but useful in other directives as well, e.g. show.

hammer2 - Add decompression support to the recovery code

* Decompress LZ4 and ZLIB-compressed data blocks when writing
out results. Was previously leaving the blocks in a compressed
form, resul

hammer2 - Add decompression support to the recovery code

* Decompress LZ4 and ZLIB-compressed data blocks when writing
out results. Was previously leaving the blocks in a compressed
form, resulting in numerous garbage-looking files.

show more ...

hammer2 - Add "hammer2 recover" directive, recover/undo single file, part 1

* Adds the 'hammer2 recover' directive. The initial commit only allows
a single filename to be recovered. You specify

hammer2 - Add "hammer2 recover" directive, recover/undo single file, part 1

* Adds the 'hammer2 recover' directive. The initial commit only allows
a single filename to be recovered. You specify the filename, not a
path.

A future version of this directive will be capable of recovering
whole directory structures. This initial version can be used to
recover important specific files that you absolutely need to undo
or restore.

* This directive requires root access because it does a raw scan of the
hammer2 media. The media will be scanned in 2 passes, so a recovery
operation can take a while.

The operation may be executed on live media. However, if trying to
restore a deleted file you might want to temporarily disable the
hammer2 periodics until recovery is done to ensure that a bulkfree
operation is not executed during the restoral. This will improve
your chances of restoring the lost files.

Note that the scan does not currently make the distinction between
data blocks and meta-data blocks. It scans everything under the
assumption that the media might be corrupt. Thus it is possible
for user data to be misinterpreted as meta-data. However, even
though this is possible, the scan does CRC checks and further
validation so in all likelihood user data will not confuse it.

* In pass 1, the operation scans the media for stuff that looks like
directory entries. If the filename is found, the inode number is
recorded in pass 1.

In pass 2, searches the media for blockref entries for inodes matching
any inode number found in pass 1. Matching entries are restored
as "<destdir>/<filename>.<sequence_number>", %05ld so
.00001, .00002, etc.

Multiple versions of the same file with the same inode number,
multiple versions of the same file with different inode numbers,
and different files with the same filename, will all be restored.

* The recovery operation is able to validate the full content of the
inode. If any content is found to be corrupted, as much of the file
as possible will be restored but the file will be renamed
"<filename>.<seqno>.corrupted" and perms will not be restored.
Ownership and mtime will still be restored.

show more ...

sys/vfs/hammer2: Fix -Wpointer-sign warnings on hammer2_dirhash()

Warned on Linux user space.

Change hammer2_dirhash() and user space dirhash() to take
const char* instead of const unsigned char* f

sys/vfs/hammer2: Fix -Wpointer-sign warnings on hammer2_dirhash()

Warned on Linux user space.

Change hammer2_dirhash() and user space dirhash() to take
const char* instead of const unsigned char* for entry "name".

* hammer2_dirhash() callers always pass char* or const char*,
except for one in sbin/newfs_hammer2/mkfs_hammer2.c where filename[]
in ondisk inode (unterminated unsigned char array) is used.

* hammer2_dirhash() only compares name with ascii characters,
and passes substring to hammer2_icrc32() which takes const void*,
so it doesn't need to be unsigned.

* Also change hammer2_inode_create_pfs() (another function whose
callers only pass char* or const char* for PFS "name") to take
const char* instead of const uint8_t* so that hammer2_dirhash()
doesn't get warned after changes above.

show more ...

sys/vfs/hammer2: Add initial multi-volumes support for HAMMER2

This commit adds initial multi-volumes support for HAMMER2. Maximum
supported volumes is 64. The feature and implementation is similar

sys/vfs/hammer2: Add initial multi-volumes support for HAMMER2

This commit adds initial multi-volumes support for HAMMER2. Maximum
supported volumes is 64. The feature and implementation is similar to
multi-volumes support in HAMMER1.

1. ondisk changes
=================
This commit bumps volume header version from 1 to 2, and adds four new
volume header fields using reserved fields in version 1. Other ondisk
structures are unchanged.
* "volu_id" - volume id from 0 to 63, where 0 represents root volume.
* "nvolumes" - number of volumes. All volumes have same the same value.
* "total_size" - sum of "volu_size" in volumes. All volumes have the
same value.
* "volu_loff[HAMMER2_MAX_VOLUMES]" - A 512 bytes table which contains
start offset of max 64 volumes within "total_size". All volumes have
the same value.

Version 1 volume header has 0 for above fields, so HAMMER2 internally
treats "nvolumes" as 1, and "total_size" as "volu_size" to be able to
handle version 1 and 2 transparently.

All volumes have 4 headers, but only root volume ones are relevant.
Non-root volume headers have their own unique "volu_id" and "volu_size",
but other fields are unimportant and never used. Non-root volume headers
have sroot blockset[i] whose type is HAMMER2_BREF_TYPE_INVALID. Non-root
volume headers don't have boot/aux area, so freemap area start from
offset 0. Non-root volume headers are readonly and never updated after
creation. This means non-root volumes are just extra storage to extend
fs size and internally make up a single virtual volume whose size is
"total_size".

It currently doesn't automatically upgrade an existing version 1 fs to
version 2. Only newly created fs becomes version 2 for now.

2. volumes layout
=================
Basically similar to HAMMER1. A first block device argument provided for
newfs_hammer2(8) becomes the root volume, and if specified remaining
devices extend "total_size" as non-root volumes. All volumes except for
the last one have 1GiB (freemap level1) aligned "volu_size".

This means each volume's start offset within "total_size" is also 1GiB
(freemap level1) aligned. The start offsets of volumes are stored in
volu_loff[HAMMER2_MAX_VOLUMES]. Each volu_loff[n] (0 <= n < nvolumes)
represents start offset of volume n within "total_size". Unused volumes
have -1 for volu_loff[n].
e.g. If a fs consists of 1 volume, volu_loff[0] has 0 and rests have -1.
e.g. If a fs consists of 3 volumes, x GiB root volume, y GiB volume,
and z GiB volume, volu_loff[0] has 0, volu_loff[1] has x, volu_loff[2]
has x+y, and rests have -1.

Low level I/O function in HAMMER2 uses this linear offsets table to
determine a device vnode to use and relative offset within the device
vnode, for a given blockref's "data_off". This is different from HAMMER1
where logical offset had embedded volume id bits (i.e. there were holes
in logical address space). HAMMER2 needs this table to support multi-
volumes without changing current logical offset mechanism.

Unless all volumes are specified and mountable, mount_hammer2(8) fails
like it failed in HAMMER1. This also applies to other userspace commands
which require volumes specification, except for fstyp(8).

3. userspace commands
=====================
Basically same as or similar to HAMMER1.
* newfs_hammer2(8) takes a list of block device paths as argv[].
* mount_hammer2(8) takes block device paths or names in "a:b:c:..."
format.
* hammer2(8) takes block device paths or names in "a:b:c:..." format for
directives which require volumes specification. This commit also adds
"volume-list" directive and an ioctl command HAMMER2IOC_VOLUME_LIST,
which are similar to the one in HAMMER1.
* fsck_hammer2(8) takes device paths or names in "a:b:c:..." format.
* fstyp(8) takes device paths in "path1:path2:path3:..." format.

4. limitations
==============
* hammer2(8) "info" directive ignores multi-volumes block devices.
* hammer2(8) "growfs" directive doesn't support multi-volumes fs.
* fstyp(8) is unable to find PFS label via -l option if the PFS inode or
its parent indirect blocks are located beyond root volume.
* hammer2(8) doesn't support "volume-add" and "volume-del" directives
which existed in HAMMER1, and there is currently no plan to support.

show more ...

sbin/hammer2: Change hammer2_uuid_to_str() to take const pointer

sbin/hammer2: Add hammer2_pfssubtype_to_str()

hammer2 - Enhance pfs-list and pfs-delete

* Enhance pfs-list to list PFSs available across all mounted hammer2
filesystems instead of just the current directory's mount. A
specific mount may be

hammer2 - Enhance pfs-list and pfs-delete

* Enhance pfs-list to list PFSs available across all mounted hammer2
filesystems instead of just the current directory's mount. A
specific mount may be specified via -s mountpt.

* Enhance pfs-delete to look for the PFS name across all mounted
hammer2 filesystems instead of just the current directory's mount.

As a safety, pfs-delete will refuse to delete PFS names which are
duplicated across multiple mounts. A specific mount may be specified
via -s mountpt.

show more ...

sbin/hammer2: Add hammer2_breftype_to_str()

sbin/hammer2: Add sbin/hammer2/hammer2_subs.h

Separate a header for subs.c from <hammer2.h>.

This lets other HAMMER2 binaries drop unneeded dependencies
required to use sbin/hammer2/subs.c (various

sbin/hammer2: Add sbin/hammer2/hammer2_subs.h

Separate a header for subs.c from <hammer2.h>.

This lets other HAMMER2 binaries drop unneeded dependencies
required to use sbin/hammer2/subs.c (various unneeded OpenSSL
header includes via <dmsg.h> via <hammer2.h>, global variables
via <hammer2.h>).

This doesn't affect existing files which include <hammer2.h>.

show more ...