usr.sbin/makefs/hammer2: Support inode timestamp in "-o R" option

which was missing part in 5e8b0eb766966577a1da788053d7ccaa443fa222.

Get timestamp from source inode within image file, and use that

usr.sbin/makefs/hammer2: Support inode timestamp in "-o R" option

which was missing part in 5e8b0eb766966577a1da788053d7ccaa443fa222.

Get timestamp from source inode within image file, and use that to
update newly created directory and regular file.

If the underlying file system of destination files doesn't support
timestamp and/or updating fails, the failure is simply ignored.

show more ...

usr.sbin/makefs: Add HAMMER2 offline PFS get|lookup|create|delete|snapshot support

The offline PFS takes HAMMER2 specific "-o P" option.
`image-file` argument is a valid HAMMER2 image file or block

usr.sbin/makefs: Add HAMMER2 offline PFS get|lookup|create|delete|snapshot support

The offline PFS takes HAMMER2 specific "-o P" option.
`image-file` argument is a valid HAMMER2 image file or block device.
`directory` argument is unused, but needs to be a valid path or "--".

e.g.
$ makefs -t hammer2 -o P=get /dev/adx --
$ makefs -t hammer2 -o P=lookup:name /path/to/hammer2.img --
$ makefs -t hammer2 -o P=create:name /dev/adx --
$ makefs -t hammer2 -o P=delete:name /path/to/hammer2.img --
$ makefs -t hammer2 -o P=snapshot:name /dev/adx --
$ makefs -t hammer2 -o P=snapshot /path/to/hammer2.img --

The "-o P" option requires PFS command name argument, which are
"get" (alias "list"), "lookup", "create", "delete" and "snapshot".
lookup, create and delete take `:<pfs_name>` string after command name.
snapshot takes optional `:<pfs_name>` string after command name.

Use -m option (default "DATA") to specify target PFS for snapshot.

show more ...

usr.sbin/makefs: Add HAMMER2 offline bulkfree support

Since makefs HAMMER2 implements the entire HAMMER2 logic in userspace
with selected vops using single threaded xops, it's actually trivial
to su

usr.sbin/makefs: Add HAMMER2 offline bulkfree support

Since makefs HAMMER2 implements the entire HAMMER2 logic in userspace
with selected vops using single threaded xops, it's actually trivial
to support other operations, e.g. HAMMER2 ioctls.

This commit adds bulkfree (free unreferenced blocks by scanning the
entire data chains from vchain) option to makefs. Unlike the existing
hammer2(8) "bulkfree" directive which requires live filesystem, this
option enables offline bulkfree against unmounted HAMMER2 image.

The offline bulkfree takes HAMMER2 specific "-o B" option. When this
option is specified, makefs runs offline bulkfree against `image-file`
argument instead of creating one, hence it must be a valid HAMMER2
image. `image-file` can be either a regular file or block device.
Unlike normal use case, `directory` argument is unused, but it's
still required. It can be any valid path or simply "--".

e.g.
$ makefs -t hammer2 -o B /dev/adx --
$ makefs -t hammer2 -o B /path/to/hammer2.img --

Technically, all HAMMER2 ioctls can be implemented in makefs as
offline version, but "bulkfree" and "growfs" are probably the only
hammer2(8) directives that make sense to exist as offline version.

Note that the limitation regarding OOM mentioned in 2d60b848f2 also
applies to bulkfree, i.e. makefs(8) could fail with partially written
`image-file` if it contains insane number of files or directories.

show more ...

usr.sbin/makefs: Sync with sys/vfs/hammer2

Sync with following sys/vfs/hammer2 commits.

$ git log --pretty="%h %s" 6966bebf98ca11fee72e5ba9849224a5049211dd.. -- sys/vfs/hammer2/
23d820f091 sys/vfs/

usr.sbin/makefs: Sync with sys/vfs/hammer2

Sync with following sys/vfs/hammer2 commits.

$ git log --pretty="%h %s" 6966bebf98ca11fee72e5ba9849224a5049211dd.. -- sys/vfs/hammer2/
23d820f091 sys/vfs/hammer2: Use howmany() to calculate bulkfree bmap size
e12df3bcd3 sys/vfs/hammer2: Fix comment on bmradix in freemap
8d92096877 sys/vfs/hammer2: Remove obsolete comments in freemap
f517f709f2 sys/vfs/hammer2: Remove redundant NULL chain test in bulkfree
c4cc6dbe36 sys/vfs/hammer2: Remove obsolete code/comment in freemap
a4cea70ee4 sys/vfs/hammer2: Fix 53b767003a to use plural
53b767003a sys/vfs/hammer2: Fix multiple "radii" -> "radix"

show more ...

usr.sbin/makefs/hammer2: Avoid #include "makefs.h" in hammer2.h

Not a build error from ddd1d3d1ff2fecb3b58ce152a65095abaf634491,
but safer to avoid including "makefs.h" in usr.sbin/makefs/hammer2/ha

usr.sbin/makefs/hammer2: Avoid #include "makefs.h" in hammer2.h

Not a build error from ddd1d3d1ff2fecb3b58ce152a65095abaf634491,
but safer to avoid including "makefs.h" in usr.sbin/makefs/hammer2/hammer2.h
which basically shouldn't know anything about makefs(8).

show more ...

usr.sbin/makefs/hammer2: Derive timestamp from source inodes

makefs(8) HAMMER2 has been using current time for inode's ctime/mtime.
Change hammer2_update_time() to use current fsnode's timestamp.
Al

usr.sbin/makefs/hammer2: Derive timestamp from source inodes

makefs(8) HAMMER2 has been using current time for inode's ctime/mtime.
Change hammer2_update_time() to use current fsnode's timestamp.
Also support -T option.

Note that both hammer2_curnode (current fsnode) and stampst (-T option)
are global variables.

show more ...

usr.sbin/makefs: Sync with sys/vfs/hammer2

Sync with following sys/vfs/hammer2 commits.

$ git log --pretty="%h %s" 66f2c621b4a3c452381030521ca3bb6dadfba431.. -- sys/vfs/hammer2/
1d5238a4ed sys/vfs/

usr.sbin/makefs: Sync with sys/vfs/hammer2

Sync with following sys/vfs/hammer2 commits.

$ git log --pretty="%h %s" 66f2c621b4a3c452381030521ca3bb6dadfba431.. -- sys/vfs/hammer2/
1d5238a4ed sys/vfs/hammer2: Fix -Wpointer-sign warnings on hammer2_dirhash()
1b796c7004 sys/vfs/hammer2: Fix -Wpointer-sign warnings

show more ...

usr.sbin/makefs: Sync with sys/vfs/hammer2

Sync with following sys/vfs/hammer2 commits.

$ git log --pretty="%h %s" 185ae703d726eb41fe0f44e432a9575d06b2ba45.. -- sys/vfs/hammer2/
ae4718daaa sys/vfs/

usr.sbin/makefs: Sync with sys/vfs/hammer2

Sync with following sys/vfs/hammer2 commits.

$ git log --pretty="%h %s" 185ae703d726eb41fe0f44e432a9575d06b2ba45.. -- sys/vfs/hammer2/
ae4718daaa sys/vfs/hammer2: Remove redundant "spmp->iroot = NULL;"
ee92f60939 sys/vfs/hammer2: Fix many comments
be41253d2a sys/vfs/hammer2: Rename HAMMER2_BREF_FLAG_ZERO -> HAMMER2_BREF_FLAG_UNUSED
64eadfb368 sys/vfs/hammer2: Fix incorrect reservedXXX names in ondisk structure
6feb58ab26 sys/vfs/hammer2: Minor cleanups for hammer2_disk.h
4b2ec0813c sys/vfs/hammer2: Move HAMMER2_{CHECK,COMP}_STRINGS to sbin/hammer2
cca587dd52 sys/vfs/hammer2: Move HAMMER2_METH_DEFAULT from hammer2_disk.h to hammer2.h
d21704d78e sys/vfs/hammer2: Fix a typo in a source code comment
fac0c41fdd sys/vfs/hammer2: Fix "already initialized" multi-volumes error message
6763396fd5 sys/vfs/hammer2: Remove redundant error check in hammer2_vfs_mount()
12757b74e2 sys/vfs/hammer2: Make fchain/vchain dump on last drop consistent
a011271d2d sys/vfs/hammer2: Remove comments on removed stuff
e6a1580946 sys/vfs/hammer2: Remove obsolete comment on HAMMER2_RESOLVE_RDONLY
e8e30c7644 sys/vfs/hammer2: Remove obsolete comment in hammer2_vop_readdir()
25b8f48546 sys/vfs/hammer2: #if0 hammer2_chain_lock_unhold()/hammer2_chain_unlock_hold()
b5513ad07d sys/vfs/hammer2: Use correct XOP struct in hammer2_xop_{lookup,delete}
5b691df11e sys/vfs/hammer2: hammer2_dump_chain() should belong to hammer2_chain.c
92931eab21 sys/vfs/hammer2: Use HAMMER2_PFSTYPE_NONE rather than 0
16f0b63d7f sys/vfs/hammer2: Make sure PFS exists after chain lookup on mount
4b0205e630 sys/vfs/hammer2: hammer2_ioctl_inode_{get,set} never returns error
634b6e30ca sys/vfs/hammer2: Add include guard for hammer2_xxhash.h
bc7ea12e61 sbin/hammer2: Remove HAMMER1 comment from dirhash()
47f63aa5d3 sys/vfs/hammer2: Rename hammer2_chain_core_init() -> hammer2_chain_init()
507074cda8 sys/vfs/hammer2: Change readonly purpose sysctls to CTLFLAG_RD
39b691836e sys/vfs/hammer2: Use __debugvar for RB_INSERT() result only used in KKASSERT
a4959f8900 sys/vfs/hammer2: Use __debugvar rather than trying to use it outside KASSERT
9944926ceb sys/vfs/hammer2: Avoid unused warning when !INVARIANTS

show more ...

usr.sbin/makefs: Add HAMMER2 support

This commit adds HAMMER2 image creation support for makefs(8).
It runs newfs_hammer2(8) and then sys/vfs/hammer2 logic in userspace
to create HAMMER2 image from

usr.sbin/makefs: Add HAMMER2 support

This commit adds HAMMER2 image creation support for makefs(8).
It runs newfs_hammer2(8) and then sys/vfs/hammer2 logic in userspace
to create HAMMER2 image from a given directory.

This commit splits newfs_hammer2(8) into newfs and mkfs part simlarly
to newfs_msdos(8), so that makefs(8) can use newfs functionality.
The entire sys/vfs/hammer2 (with exception of unneeded
hammer2_{bulkfree,ccms,iocom,ioctl,msgops,synchro}.[hc] and reusable
hammer2_disk.h) is copied to usr.sbin/makefs with below modification.
It intends to have minimum amount of diff against sys/vfs/hammer2.

* Header includes are modified so that it compiles in userspace.
* VFS and other kernel functions are usually implemented as simple
stub functions in hammer2_compat.h and hammer2_buf.c, but some are
commented out.
* Kernel functions such as kprintf, kmalloc, kprintf, kstrdup, etc
are implemented using corresponding libc functions.
* Lock primitives are basically NOP, and they (should) never block
as makefs(8) is a single thread program.
* struct vnode and struct buf (the ones defined locally in makefs(8),
not sys/sys/*) have new struct members only used by HAMMER2 to
emulate VFS behavior required by HAMMER2.
* Since makefs(8) is write-only, VOP_{NRESOLVE,NCREATE,NMKDIR,NLINK,
NSYMLINK,WRITE,STRATEGY} are implemented, but other VOPs just
return EOPNOTSUPP.
* VOP_{INACTIVE,RECLAIM} may be implemented and used in future to
better emulate VFS behavior to address current limitation.
* VOP_WRITE is modified to directly call VOP_STRATEGY function.
* The XOP kernel thread is modified to act as a regular function
called from VOPs, along with simplified admin code.

It currently has following limitations.

* multi-volumes is unsupported, simply due to makefs(8) only taking 1
image file path.
* Not necessarily a limitation, but it only supports populating 1 PFS,
which is "DATA" by default. Other PFSes if any won't have anything
under the root PFS inode.
* makefs(8) process gets killed by OOM for a directory with *extremely*
large number of files, depending on available memory. This is due to
the way it currently tries to flush all chains in a single VFS_SYNC.
Supporting multiple VFS_SYNC calls by checking available memory along
the way gives chance to free unused vnodes/inodes and chains. This
may be implemented in future. This limitation is specific to HAMMER2,
as all other makefs(8) filesystems are not CoW, meaning they allow
in-place write based objects creation from a top directory to bottom
whereas HAMMER2 flushes chains in bottom-up direction.

show more ...