Use .Fn for function names in some manual pages.

Remove various 'r' commands and daemons.

rcp(1), rlogin(1), rlogind(1), rsh(1) and rshd(1). In the unlikely case
that someone still needs them, they can use the net/bsdrcmds port to
bring them back.

Remove various 'r' commands and daemons.

rcp(1), rlogin(1), rlogind(1), rsh(1) and rshd(1). In the unlikely case
that someone still needs them, they can use the net/bsdrcmds port to
bring them back.

FreeBSD removed them in October 2017 and I took their work as a guideline.

show more ...

sbin/hammer: Fix hammer(8)

It's no longer HAMMER volume once stripped, so drop "HAMMER".

hammer - HAMMER Version 7

* Add support for version 7 which changes the CRC mechanic from the
old slow CRC code to the faster ISCSI CRC code. We don't use the CRC
instruction yet but ths base I

hammer - HAMMER Version 7

* Add support for version 7 which changes the CRC mechanic from the
old slow CRC code to the faster ISCSI CRC code. We don't use the CRC
instruction yet but ths base ISCSI CRC from FreeBSD is 6x faster than
the CRC code we were using before.

* Change newfs_hammer default to version 7 (for master).

show more ...

Fix two typos in manual pages.

sbin/hammer: Don't hardcode 0 for root PFS

HAMMER code doesn't hardcode 0 for root PFS
(e.g. see sbin/newfs_hammer, it could be !=0 if one wants to do so).

Fix the existing error messages using har

sbin/hammer: Don't hardcode 0 for root PFS

HAMMER code doesn't hardcode 0 for root PFS
(e.g. see sbin/newfs_hammer, it could be !=0 if one wants to do so).

Fix the existing error messages using hardcoded 0.
Also add "(root PFS)" for PFS#0 in hammer info command.

show more ...

sbin/hammer: Add full mode for hammer recover to revive full scan

This commit revives the original full scan recovery by adding full
option, after the previous commit introduced offset limit.
Appare

sbin/hammer: Add full mode for hammer recover to revive full scan

This commit revives the original full scan recovery by adding full
option, after the previous commit introduced offset limit.
Apparently, both full option and quick option can't be specified.

To summarize 3 modes,

1. default - Full scan, but only upto the last big-block being used.
2. full - Full scan, which scans the entire fs image with no limit.
3. quick - B-Tree only scan, plus associated records in other zones.

1. was introduced (by the previous commit) to fix a bug, as well as
to avoid irrelevant files.
2. was introduced (by this commit) to revive the original full scan
recovery behavior, which is by far the slowest, but most reliable
in terms of recovery except for the above bug.
3. was introduced (by e819b271) to speed up the recovery process,
provided B-Tree zone is not corrupted. This is the fastest.

show more ...

sbin/hammer: Add quick mode for hammer recover

Since hammer recover command tries to recover filesystem data
based on assumption on ondisk data bytes that look like B-Tree
nodes/elms, the command ca

sbin/hammer: Add quick mode for hammer recover

Since hammer recover command tries to recover filesystem data
based on assumption on ondisk data bytes that look like B-Tree
nodes/elms, the command can tell the recovery process is done
once scanning offset gets to the point where there is no more
big-blocks for B-Tree zone, without scanning through the whole
address space of all volumes (provided B-Tree zone is alive).

By specifying quick option after the target directory option,
this command makes use of B-Tree big-block info prefetched before
recovery process, and stops recovery once all B-Tree big-blocks
are scanned. As shown in below example, this makes recovery
much faster by cutting unnecessary I/Os.

The drawback is that quick mode is based on assumption that
B-Tree zone isn't corrupted. If B-Tree zone is somehow corrupted,
prefetched info is incomplete or totally wrong, so one needs
to linearly scan the whole address space of all volumes to
check every possible B-Tree nodes/elms without using quick mode
which is what's been done by default.

-- example of default and quick mode
# newfs_hammer -L TEST /dev/da1 > /dev/null
# mount_hammer /dev/da1 /HAMMER
# cd /HAMMER
# git clone /usr/local/src/dragonfly > /dev/null 2>&1
# cd
# umount /HAMMER

# time hammer -f /dev/da1 recover /tmp/a > /dev/null
hammer -f /dev/da1 recover /tmp/a > /dev/null 309.51s user 122.96s system 21% cpu 33:50.17 total
# cd /tmp/a/PFS00000/dragonfly/sys/vfs/hammer
# make > /dev/null 2>&1; echo $?
0
# file hammer.ko
hammer.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

# time hammer -f /dev/da1 recover /tmp/b quick > /dev/null
hammer -f /dev/da1 recover /tmp/b quick > /dev/null 0.41s user 3.41s system 14% cpu 26.652 total
# cd /tmp/b/PFS00000/dragonfly/sys/vfs/hammer
# make > /dev/null 2>&1; echo $?
0
# file hammer.ko
hammer.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

show more ...

sbin/hammer: Add hammer strip command

This command is inspired by hammer recover command, and does
opposite of what recover command does.

This command zero clears zone-8(B-Tree) big-blocks, zone-9(

sbin/hammer: Add hammer strip command

This command is inspired by hammer recover command, and does
opposite of what recover command does.

This command zero clears zone-8(B-Tree) big-blocks, zone-9(meta)
big-blocks, and then the whole volume header, except that volume
signature field is overwritten with "STRIPPED" instead of zeros.
After running, a filesystem is no longer mountable or recoverable
with hammer recover command. This command is also fast as it only
zero clears good enough ondisk data to make it unmountable and
unrecoverable.

Keep in mind that this command does _not_ zero clear user data.
Users would normally use a software designed to completely shred
a filesystem. This command is not designed to shred a filesystem.
The name "strip" gives better idea of what it really does than
using "shred"/etc.

-- example
# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# cd /HAMMER
# dd if=/dev/urandom of=./out bs=1M count=120000
120000+0 records in
120000+0 records out
125829120000 bytes transferred in 1766.417077 secs (71234094 bytes/sec)
# cd
# umount /HAMMER
# hammer -f /dev/da1:/dev/da2:/dev/da3 strip
You have requested that HAMMER filesystem (TEST) be stripped
Do you really want to do this? [y/n] y
Stripping HAMMER filesystem (TEST) in 5 4 3 2 1.. starting destruction pass
8000000021000000
9000000021800000
800000019c000000
800000030c000000
800000047e000000
80000005f7000000
8000000767000000
80000008d8000000
8000000a51800000
8000000bc5000000
8000000d37800000
8000000ead000000
800000101e800000
8000001193000000
8000001304000000
8000001478800000
80000015ee000000
8000001760800000
80000018d1800000
8000001a47000000
8000001bb6000000
801000013c000000
/dev/da1
/dev/da2
/dev/da3
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
mount: Invalid argument
mount_hammer: /dev/da1: Invalid volume signature 4445505049525453

show more ...

sys/vfs/hammer: Fix terminology for undo fifo

Always use "UNDO/REDO FIFO" whenever printf/manpage/etc refer to
undo in terms of HAMMER's zone-3 data structure, because that's
how it's been used orig

sys/vfs/hammer: Fix terminology for undo fifo

Always use "UNDO/REDO FIFO" whenever printf/manpage/etc refer to
undo in terms of HAMMER's zone-3 data structure, because that's
how it's been used originally.

But leave comments as they are now. There are tons of comments
that refer to UNDO, but not REDO when they could be UNDO/REDO,
but those mostly come from the fact that REDO was added later.

show more ...

sbin/hammer: Change "PFS #" to "PFS#"

It's mostly "PFS#", so remove extra space between "PFS" and "#".

sbin/hammer: Fix hammer(8) manpage

See sbin/hammer/cmd_cleanup.c.

# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# hammer cleanup

sbin/hammer: Fix hammer(8) manpage

See sbin/hammer/cmd_cleanup.c.

# newfs_hammer -L TEST /dev/da1 /dev/da2 /dev/da3 > /dev/null
# mount_hammer /dev/da1:/dev/da2:/dev/da3 /HAMMER
# hammer cleanup /HAMMER > /dev/null
# hammer config /HAMMER
snapshots 1d 60d
prune 1d 5m
rebalance 1d 5m
#dedup 1d 5m
reblock 1d 5m
recopy 30d 10m
# umount /HAMMER
# hammer -vvvf /dev/da1:/dev/da2:/dev/da3 show | grep config -A7
config text="
snapshots 1d 60d
prune 1d 5m
rebalance 1d 5m
#dedup 1d 5m
reblock 1d 5m
recopy 30d 10m
"

show more ...

sbin/hammer: Make hammer commands print root volume path

Base on the change made by the previous commit, this commit adds
the root volume path information (not root volume#) to hammer info
and hamme

sbin/hammer: Make hammer commands print root volume path

Base on the change made by the previous commit, this commit adds
the root volume path information (not root volume#) to hammer info
and hammer volume-list commands. hammer volume-list shows the path
only on -v. This enables users to explicitly know the root volume.

[A] The existing /sbin/hammer
# hammer -v volume-list /HAMMER
/dev/da1
/dev/da2
/dev/da3
# hammer info /HAMMER | grep Root
Root Volume 0

[B] With this commit
# ./hammer -v volume-list /HAMMER
0 /dev/da1 (Root Volume)
1 /dev/da2
2 /dev/da3
# hammer info /HAMMER | grep Root
Root Volume /dev/da1

show more ...

sbin/hammer: Minor fix for hammer(8) manpage

sbin/hammer: Fix hammer(8) manpage

Add a sentence that explains PFS via PFS is not allowed.

hammer.8: Use full month name in .Dd

Fix various typos in manual pages.

sbin/hammer: Fix and change hammer show behavior when only lo is used

913b6663 has expanded hammer show's B-Tree directive from
lo:objid
to
lo:objid:rectype:key:tid
which allowed hammer show to filt

sbin/hammer: Fix and change hammer show behavior when only lo is used

913b6663 has expanded hammer show's B-Tree directive from
lo:objid
to
lo:objid:rectype:key:tid
which allowed hammer show to filter B-Tree iteration in the
same way that the actual filesystem does on B-Tree lookup.

There was an exception in 913b6663 where one could not just
use lo directive, but was forced to use at least the first
two directives lo:objid. If objid wasn't specified then
HAMMER_MIN_OBJID was used for objid by default.

This was to keep compatibility with old behavior before
913b6663 added the rest of the directives (rectype:key:tid),
but this old behavior wasn't really useful for anything and
rather confusing given that all unspecified fields (after
the last specified field) except for objid were wildcard.
See sbin/hammer/hammer.c and sbin/hammer/cmd_show.c before
913b6663 (or really old one like v4.0 era) for details.

This commit normalizes this exceptional behavior and make
all option directives behave the same way. For example, if
*only* lo field is specified, then the rest of the fields
are wildcard. If the first n fields are specified, then the
remaining (5-n) fields are wildcard.

This does change the behavior when *only* lo is specified,
but the change should make the behavior less confusing.

Also update hammer(8) manpage since it hasn't been updated.

show more ...

hammer: Print "B-Tree" (conform to hammer specification)

Hammer specification, manpage, comments, etc mostly use
"B-Tree" instead of "btree" or "b-tree".

This commit replaces "btree" and "b-tree" s

hammer: Print "B-Tree" (conform to hammer specification)

Hammer specification, manpage, comments, etc mostly use
"B-Tree" instead of "btree" or "b-tree".

This commit replaces "btree" and "b-tree" strings that appear
in printf() and hammer(8) with "B-Tree", assuming no user or
userspace triggers anything by these strings.

show more ...

hammer: Cleanup hammer(8) manpage

hammer(8) uses "PFS" but not "pfs" unless "pfs" appears in a path.

sbin/hammer: Make hammer history support per-path @offset,length

This commit allows each arg of the hammer history to take
@offset,length attribute. It was strange behavior that the
command takes a

sbin/hammer: Make hammer history support per-path @offset,length

This commit allows each arg of the hammer history to take
@offset,length attribute. It was strange behavior that the
command takes a global @offset,length attribute while the
command supports multiple paths to lookup history for.

This commit doesn't break the existing behavior where one
specifies @offset,length attribute as a part of the command
name as "history@offset,length".

If the old style attribute (history@offset,length) is used,
then the command doesn't parse per-path attribute, but sees
the whole string as a path. Otherwise the command parses
per-path attribute and uses offset,length unique to each,
unless the whole string including the attribute part exists
as a path (can stat(2)).

=====
# touch /HAMMER/out1
# touch /HAMMER/out2
# for x in {1..5}; do
> cat dragonfly/sys/vfs/hammer/hammer.h >> /HAMMER/out1
> sync; sync; sync; sync; sleep 1
> done
# for x in {1..5}; do
> cat dragonfly/sys/vfs/hammer/hammer.h >> /HAMMER/out2
> sync; sync; sync; sync; sleep 1
> done
# hammer history /HAMMER/out1 /HAMMER/out2
/HAMMER/out1 0000000184acac5e clean {
0000000184b59b00 15-Sep-2015 03:56:46
0000000184b59b20 15-Sep-2015 03:56:47
0000000184b59c80 15-Sep-2015 03:56:48
0000000184b59dc0 15-Sep-2015 03:56:49
0000000184b59f20 15-Sep-2015 03:56:50
0000000184b5a060 15-Sep-2015 03:56:51
}
/HAMMER/out2 0000000184acaca6 clean {
0000000184b59b00 15-Sep-2015 03:56:46
0000000184b5a1c0 15-Sep-2015 03:57:02
0000000184b5a320 15-Sep-2015 03:57:03
0000000184b5a460 15-Sep-2015 03:57:04
0000000184b5a5c0 15-Sep-2015 03:57:05
0000000184b5a700 15-Sep-2015 03:57:06
}
# hammer history@50000,1000 /HAMMER/out1 /HAMMER/out2
/HAMMER/out1 0000000184acac5e clean {
0000000184b59b20 15-Sep-2015 03:56:47
0000000184b59c80 15-Sep-2015 03:56:48
}
/HAMMER/out2 0000000184acaca6 clean {
0000000184b5a1c0 15-Sep-2015 03:57:02
0000000184b5a320 15-Sep-2015 03:57:03
}
# hammer history /HAMMER/out1@50000,1000 /HAMMER/out2@50000,1000
/HAMMER/out1 0000000184acac5e clean {
0000000184b59b20 15-Sep-2015 03:56:47
0000000184b59c80 15-Sep-2015 03:56:48
}
/HAMMER/out2 0000000184acaca6 clean {
0000000184b5a1c0 15-Sep-2015 03:57:02
0000000184b5a320 15-Sep-2015 03:57:03
}
# hammer history /HAMMER/out1@50000,1000 /HAMMER/out2@200000,1000
/HAMMER/out1 0000000184acac5e clean {
0000000184b59b20 15-Sep-2015 03:56:47
0000000184b59c80 15-Sep-2015 03:56:48
}
/HAMMER/out2 0000000184acaca6 clean {
0000000184b5a5c0 15-Sep-2015 03:57:05
}
# hammer history@50000,1000 /HAMMER/out1@50000,1000 /HAMMER/out2@200000,1000
/HAMMER/out1@50000,1000 No such file or directory
/HAMMER/out2@200000,1000 No such file or directory
# touch /HAMMER/out1@50000,1000
# sync; sync; sync; sync; sleep 1
# touch /HAMMER/out2@200000,1000
# sync; sync; sync; sync; sleep 1
# hammer history /HAMMER/out1@50000,1000 /HAMMER/out2@200000,1000
/HAMMER/out1@50000,1000 0000000184acaca7 clean {
0000000184b6ae20 15-Sep-2015 04:05:09
}
/HAMMER/out2@200000,1000 0000000184acae20 clean {
0000000184b6b180 15-Sep-2015 04:06:11
}

show more ...

sbin/hammer: Add hammer stats command (= bstats + iostats)

It's simply good to have a command that combines these two.
One often runs these two concurrently and tries to see both
at the same time an

sbin/hammer: Add hammer stats command (= bstats + iostats)

It's simply good to have a command that combines these two.
One often runs these two concurrently and tries to see both
at the same time anyway.

show more ...

hammer2 - Refactor bulkfree

* Change the bulkfree scan from breadth-first to depth-first. This
improves disk performance significantly (~2x) and is also needed for the
duplicate-detection featu

hammer2 - Refactor bulkfree

* Change the bulkfree scan from breadth-first to depth-first. This
improves disk performance significantly (~2x) and is also needed for the
duplicate-detection feature.

* Create an 8-way set-associative hash table similar to what the live
dedup code uses. Record the data_off for elements we have processed
and detect if a duplicate is encountered so we do not have to re-process
the duplicate subtree.

Also prioritize the table based on the aggregate bottom-up inode count
to reduce the chance that a top-level duplicate (aka snapshot) will get
kicked out of the hash table.

* Clean up the hammer2_chain_scan() API, making it more bref-centric
which allows us to avoid instantiating chain structures for leaf
entities. This significantly improves performance and increases
flexibility.

* Manual page adjustments for kern.maxvnodes settings suggestions.

show more ...

sbin/newfs_hammer: Explicitly tell which volume is root volume

hammer volume-del says "Cannot remove root-volume" if one
tries to remove the root volume, however no hammer commands
actually tell whi

sbin/newfs_hammer: Explicitly tell which volume is root volume

hammer volume-del says "Cannot remove root-volume" if one
tries to remove the root volume, however no hammer commands
actually tell which one is the root volume.

This commit makes newfs_hammer printf the root volume name,
followed by the existing undo/etc info of the root volume.
Also refer to the root volume in manpages.

(It may be helpful if other hammer commands like info or
volume-list show the root volume name just like newfs_hammer,
however it's not too trivial to do that. One reason is because
hammer seems to have been designed that way so users don't
need to care about any detail of underlying fs volumes.
This commit is trivial since newfs_hammer code knows which
one is the root volume)

show more ...

sys/vfs/hammer: Don't reblock when volume to delete is empty

hammer_ioc_volume_del() needs to reblock filesystem only
when the volume hammer volume-del is trying to remove is
not empty. If it's empt

sys/vfs/hammer: Don't reblock when volume to delete is empty

hammer_ioc_volume_del() needs to reblock filesystem only
when the volume hammer volume-del is trying to remove is
not empty. If it's empty (means the volume's blockmap has
nothing other than layer2 big-block in non-root volume)
it can be removed right away.

The way it always reblocks on hammer volume-del is also
counter intuitive since not everyone expects this command
to do extra effort to delete it when it's not empty, and
force one to wait for reblock to finish.

show more ...