Increase UFS/FFS maximum link count from 32767 to 65530.

The link count for a UFS/FFS inode is stored in a signed 16-bit
integer. Thus the maximum link count has been 32767.

This limit has been rec

Increase UFS/FFS maximum link count from 32767 to 65530.

The link count for a UFS/FFS inode is stored in a signed 16-bit
integer. Thus the maximum link count has been 32767.

This limit has been recently hit by the poudriere build system when
doing a ports build as it needs one directory per port and the
number of ports recently passed 32767.

A long-term solution would be to use one of the spare 32-bit fields
in the inode to store the link count. However, the UFS1 format does
not have a spare and adding the spare in UFS2 would make it hard
to make it compatible when running on older kernels that use the
original link count field. So this patch uses the much simpler
approach of changing the existing link count field from a signed
16-bit value to an unsigned 16-bit value. It has the fewest lines
of code changes. The only thing that changes is the type in the
dinode and inode structures and the definition of UFS_LINK_MAX. It
has the added benefit that it works with both UFS1 and UFS2.

It allows easy backward compatibility. Indeed it is backward
compatibility that is the primary reason to go with this approach.
If a filesystem with the new organization is mounted on an older
kernel, it still needs to work. Thus if we move the new link count
to a new field, we still need to maintain the old link count as
best as possible even when running on a kernel that knows about the
larger link counts. And we would have to carry this overhead for
the indefinite future.

If we have a new link-count field, we will have to add a new
filesystem flag to indicate that we are running with larger link
counts. We will also need to add of one of the new-feature flags
to say that we have larger link counts. Older kernels clear the
new-feature flags that they do not know about, so when a filesystem
is used on an older kernel and then moved back to a newer one, the
newer one will know that the new link counts have not been maintained
and that it will be necessary to run a full fsck on the filesystem
to correct the link counts before it can be mounted.

With this change, older kernels will generally work with the bigger
counts. While it will not itself allow the link count to exceed
32767, it will have no problem working with inodes that have a link
count greater than 32767. Since it tests that i_nlink <= UFS_LINK_MAX,
counts that are bigger than 32767 will appear negative, so will
still pass the test. Of course, if they ever drop below 32767, they
will no longer be able to exceed 32767. The one issue is if the
link count ever exceeds 65535 then it will wrap to zero and the
older kernel will be none the wiser. But this corner case is likely
to be very rare since these kernels and the applications running
on them do not expect to be able to get link counts over 32767. And
over time, the use of new filesystems on older kernels will become
rarer and rarer.

Reported-by: Mark Millard running poudriere on the ports tree
Reviewed-by: kib, olce.freebsd_certner.fr
Tested-by: Peter Holm, Mark Millard
MFC-after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D42767

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

Spacing and alignment cleanups.

No functional change.

MFC-after: 1 week

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

UFS/FFS: Migrate to modern uintXX_t from u_intXX_t.

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t. While here also migrate u_char to uint8_

UFS/FFS: Migrate to modern uintXX_t from u_intXX_t.

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t. While here also migrate u_char to uint8_t.
Where other kernel interfaces allow, migrate u_long to uint64_t.

No functional changes intended.

MFC-after: 1 week
Sponsored-by: The FreeBSD Foundation

show more ...

ufs: be more persistent with finishing some operations

when the vnode is doomed after relock. The mere fact that the vnode is
doomed does not prevent us from doing UFS operations on it while it is

ufs: be more persistent with finishing some operations

when the vnode is doomed after relock. The mere fact that the vnode is
doomed does not prevent us from doing UFS operations on it while it is
still belongs to UFS, which is determined by non-NULL v_data. Not
finishing some operations, e.g. not syncing the inode block only because
the vnode started reclamation, is not correct.

Add macro IS_UFS() which incapsulates the v_data != NULL, and use it
instead of VN_IS_DOOMED() for places where the operation completion is
important.

Reviewed by: markj, mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34072

show more ...

ufs: Rework shortlink handling to avoid subobject overflows

Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer

ufs: Rework shortlink handling to avoid subobject overflows

Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.

Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].

Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650

show more ...

Rework MOUNTED/DOING SOFTDEP/SUJ macros

Now MNT_SOFTDEP indicates that SU are active in any variant +-J, and
SU+J is indicated by MNT_SOFTDEP | MNT_SUJ combination. The reason is
that unmount will

Rework MOUNTED/DOING SOFTDEP/SUJ macros

Now MNT_SOFTDEP indicates that SU are active in any variant +-J, and
SU+J is indicated by MNT_SOFTDEP | MNT_SUJ combination. The reason is
that unmount will be able to easily hide SU from other operations by
clearing MNT_SOFTDEP while keeping the record of the active journal.

Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D29178

show more ...

vnode: move write cluster support data to inodes.

The data is only needed by filesystems that
1. use buffer cache
2. utilize clustering write support.

Requested by: mjg
Reviewed by: asomers (previo

vnode: move write cluster support data to inodes.

The data is only needed by filesystems that
1. use buffer cache
2. utilize clustering write support.

Requested by: mjg
Reviewed by: asomers (previous version), fsu (ext2 parts), mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28679

show more ...

Remove #define _KERNEL hacks from libprocstat

Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more materi

Remove #define _KERNEL hacks from libprocstat

Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.

Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition. Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.

Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D28679

show more ...

ufs_direnter: move directory truncation to ffs_vput_pair().

VOP_VPUT_PAIR() provides the hook to do the truncation right before
unlock, which is required since truncation might need to fsync(), whic

ufs_direnter: move directory truncation to ffs_vput_pair().

VOP_VPUT_PAIR() provides the hook to do the truncation right before
unlock, which is required since truncation might need to fsync(), which
itself might unlock the directory vnode.

Set new flag IN_ENDOFF which indicates that i_endoff is valid and should
be checked against inode size. Excessive size is chomped, but this
operation is advisory and failure to truncate should not result in the
failure of the main VOP.

Reviewed by: chs, mckusick
Tested by: pho
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

show more ...

Add a framework that tracks exclusive vnode lock generation count for UFS.

This count is memoized together with the lookup metadata in directory
inode, and we assert that accesses to lookup metadata

Add a framework that tracks exclusive vnode lock generation count for UFS.

This count is memoized together with the lookup metadata in directory
inode, and we assert that accesses to lookup metadata are done under
the same lock generation as they were stored. Enabled under DIAGNOSTICS.

UFS saves additional data for parent dirent when doing lookup
(i_offset, i_count, i_endoff), and this data is used later by VOPs
operating on dirents. If parent vnode exclusive lock is dropped and
re-acquired between lookup and the VOP call, we corrupt directories.

Framework asserts that corruption cannot occur that way, by tracking
vnode lock generation counter. Updates to inode dirent members also
save the counter, while users compare current and saved counters
values.

Also, fix a case in ufs_lookup_ino() where i_offset and i_count could
be updated under shared lock. It is not a bug on its own since dvp
i_offset results from such lookup cannot be used, but it causes false
positive in the checker.

In collaboration with: pho
Reviewed by: mckusick (previous version), markj
Tested by: markj (syzkaller), pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D26136

show more ...

ufs: clean up empty lines in .c and .h files

ufs: add support for lockless lookup

ACLs are not supported, meaning their presence will force the use of the old lookup.

Reviewed by: kib
Tested by: pho (in a patchset)
Differential Revisi

ufs: add support for lockless lookup

ACLs are not supported, meaning their presence will force the use of the old lookup.

Reviewed by: kib
Tested by: pho (in a patchset)
Differential Revision: https://reviews.freebsd.org/D25579

show more ...

Further evaluation of the POSIX spec for fdatasync() shows that it
requires that new data on growing files be accessible. Thus, the
the fsyncdata() system call must update the on-disk inode when the

Further evaluation of the POSIX spec for fdatasync() shows that it
requires that new data on growing files be accessible. Thus, the
the fsyncdata() system call must update the on-disk inode when the
size of the file has changed.

This commit adds another inode update flag, IN_SIZEMOD, that gets
set any time that the file size changes. If either the IN_IBLKDATA
or the IN_SIZEMOD flag is set when fdatasync() is called, the
associated inode is synchronously written to disk. We could have
overloaded the IN_IBLKDATA flag to also track size changes since
the only (current) use case for these flags are for fsyncdata(),
but it does seem useful for possible future uses to separately
track the file size changes and the inode block pointer changes.

Reviewed by: kib
MFC with: -r361785
Differential revision: https://reviews.freebsd.org/D25072

show more ...

UFS: write inode block for fdatasync(2) if pointers in inode where allocated

The fdatasync() description in POSIX specifies that
all I/O operations shall be completed as defined for synchronized

UFS: write inode block for fdatasync(2) if pointers in inode where allocated

The fdatasync() description in POSIX specifies that
all I/O operations shall be completed as defined for synchronized I/O
data integrity completion.
and then the explanation of Synchronized I/O Data Integrity Completion says
The write is complete only when the data specified in the write
request is successfully transferred and all file system
information required to retrieve the data is successfully
transferred.

For UFS this means that all pointers must be on disk. Indirect
pointers already contribute to the list of dirty data blocks, so only
direct blocks and root pointers to indirect blocks, both of which
reside in the inode block, should be taken care of. In ffs_balloc(),
mark the inode with the new flag IN_IBLKDATA that specifies that
ffs_syncvnode(DATA_ONLY) needs a call to ffs_update() to flush the
inode block.

Reviewed by: mckusick
Discussed with: tmunro
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D25072

show more ...

Convert DOINGSOFTDEP, MOUNTEDSOFTDEP, DOINGSUJ, and MOUNTEDSUJ to being
boolean expressions so that their values are not lost when assigned to
`bool' or `int' variables.

Reviewed by: Chuck Silvers

Convert DOINGSOFTDEP, MOUNTEDSOFTDEP, DOINGSUJ, and MOUNTEDSUJ to being
boolean expressions so that their values are not lost when assigned to
`bool' or `int' variables.

Reviewed by: Chuck Silvers (chs)
MFC after: 3 days
Sponsored by: Netflix

show more ...

ufs: relax an overzealous assert added in r356671

Part of i_flag can persist across a drop to hold count of 0, at which
point the vnode is taken off the lazy list. Then whoever locks and unlocks
the

ufs: relax an overzealous assert added in r356671

Part of i_flag can persist across a drop to hold count of 0, at which
point the vnode is taken off the lazy list. Then whoever locks and unlocks
the vnode can trip on the assert.

This trips over kyua running a test untarring character devices to ufs.

Reported by: lwhsu

show more ...

ufs: use lazy list instead of active list for syncer

Quota code is temporarily regressed to do a full vnode scan.

Reviewed by: jeff
Tested by: pho (in a larger patch, previous version)
Differential

ufs: use lazy list instead of active list for syncer

Quota code is temporarily regressed to do a full vnode scan.

Reviewed by: jeff
Tested by: pho (in a larger patch, previous version)
Differential Revision: https://reviews.freebsd.org/D22996

show more ...

ufs: add a setter for inode i_flag field

This will be used later to add vnodes to the lazy list.

Reviewed by: kib (previous version), jeff
Tested by: pho (in a larger patch)
Differential Revision:

ufs: add a setter for inode i_flag field

This will be used later to add vnodes to the lazy list.

Reviewed by: kib (previous version), jeff
Tested by: pho (in a larger patch)
Differential Revision: https://reviews.freebsd.org/D22994

show more ...

Augment the UFS filesystem specific print function (called by the
kernel vn_printf() routine when printing out vnodes associated with
a UFS filesystem) to also include the inode's link count, effecti

Augment the UFS filesystem specific print function (called by the
kernel vn_printf() routine when printing out vnodes associated with
a UFS filesystem) to also include the inode's link count, effective
link count, generation number, owner, group, flags, size, and for
UFS2 filesystems, the extent size.

Sponsored by: Netflix

show more ...

Revert r313780 (UFS_ prefix)

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsor

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9623

show more ...

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

show more ...

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

show more ...