usr.sbin/makefs: Add HAMMER2 offline bulkfree support

Since makefs HAMMER2 implements the entire HAMMER2 logic in userspace
with selected vops using single threaded xops, it's actually trivial
to su

usr.sbin/makefs: Add HAMMER2 offline bulkfree support

Since makefs HAMMER2 implements the entire HAMMER2 logic in userspace
with selected vops using single threaded xops, it's actually trivial
to support other operations, e.g. HAMMER2 ioctls.

This commit adds bulkfree (free unreferenced blocks by scanning the
entire data chains from vchain) option to makefs. Unlike the existing
hammer2(8) "bulkfree" directive which requires live filesystem, this
option enables offline bulkfree against unmounted HAMMER2 image.

The offline bulkfree takes HAMMER2 specific "-o B" option. When this
option is specified, makefs runs offline bulkfree against `image-file`
argument instead of creating one, hence it must be a valid HAMMER2
image. `image-file` can be either a regular file or block device.
Unlike normal use case, `directory` argument is unused, but it's
still required. It can be any valid path or simply "--".

e.g.
$ makefs -t hammer2 -o B /dev/adx --
$ makefs -t hammer2 -o B /path/to/hammer2.img --

Technically, all HAMMER2 ioctls can be implemented in makefs as
offline version, but "bulkfree" and "growfs" are probably the only
hammer2(8) directives that make sense to exist as offline version.

Note that the limitation regarding OOM mentioned in 2d60b848f2 also
applies to bulkfree, i.e. makefs(8) could fail with partially written
`image-file` if it contains insane number of files or directories.

show more ...

usr.sbin/makefs: Add HAMMER2 support

This commit adds HAMMER2 image creation support for makefs(8).
It runs newfs_hammer2(8) and then sys/vfs/hammer2 logic in userspace
to create HAMMER2 image from

usr.sbin/makefs: Add HAMMER2 support

This commit adds HAMMER2 image creation support for makefs(8).
It runs newfs_hammer2(8) and then sys/vfs/hammer2 logic in userspace
to create HAMMER2 image from a given directory.

This commit splits newfs_hammer2(8) into newfs and mkfs part simlarly
to newfs_msdos(8), so that makefs(8) can use newfs functionality.
The entire sys/vfs/hammer2 (with exception of unneeded
hammer2_{bulkfree,ccms,iocom,ioctl,msgops,synchro}.[hc] and reusable
hammer2_disk.h) is copied to usr.sbin/makefs with below modification.
It intends to have minimum amount of diff against sys/vfs/hammer2.

* Header includes are modified so that it compiles in userspace.
* VFS and other kernel functions are usually implemented as simple
stub functions in hammer2_compat.h and hammer2_buf.c, but some are
commented out.
* Kernel functions such as kprintf, kmalloc, kprintf, kstrdup, etc
are implemented using corresponding libc functions.
* Lock primitives are basically NOP, and they (should) never block
as makefs(8) is a single thread program.
* struct vnode and struct buf (the ones defined locally in makefs(8),
not sys/sys/*) have new struct members only used by HAMMER2 to
emulate VFS behavior required by HAMMER2.
* Since makefs(8) is write-only, VOP_{NRESOLVE,NCREATE,NMKDIR,NLINK,
NSYMLINK,WRITE,STRATEGY} are implemented, but other VOPs just
return EOPNOTSUPP.
* VOP_{INACTIVE,RECLAIM} may be implemented and used in future to
better emulate VFS behavior to address current limitation.
* VOP_WRITE is modified to directly call VOP_STRATEGY function.
* The XOP kernel thread is modified to act as a regular function
called from VOPs, along with simplified admin code.

It currently has following limitations.

* multi-volumes is unsupported, simply due to makefs(8) only taking 1
image file path.
* Not necessarily a limitation, but it only supports populating 1 PFS,
which is "DATA" by default. Other PFSes if any won't have anything
under the root PFS inode.
* makefs(8) process gets killed by OOM for a directory with *extremely*
large number of files, depending on available memory. This is due to
the way it currently tries to flush all chains in a single VFS_SYNC.
Supporting multiple VFS_SYNC calls by checking available memory along
the way gives chance to free unused vnodes/inodes and chains. This
may be implemented in future. This limitation is specific to HAMMER2,
as all other makefs(8) filesystems are not CoW, meaning they allow
in-place write based objects creation from a top directory to bottom
whereas HAMMER2 flushes chains in bottom-up direction.

show more ...