fusefs: annotate deliberate file descriptor leaks in the tests

closing a file descriptor causes FUSE activity that is superfluous to the
purpose of most tests, but would nonetheless require matching

fusefs: annotate deliberate file descriptor leaks in the tests

closing a file descriptor causes FUSE activity that is superfluous to the
purpose of most tests, but would nonetheless require matching expectations.
Rather than do that, most tests deliberately leak file descriptors instead.
This commit moves the leakage from each test into two trivial functions:
leak and leakdir. Hopefully Coverity will only complain about those
functions and not all of their callers.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: implement protocol 7.23's FUSE_WRITEBACK_CACHE option

As of protocol 7.23, fuse file systems can specify their cache behavior on a
per-mountpoint basis. If they set FUSE_WRITEBACK_CACHE in

fusefs: implement protocol 7.23's FUSE_WRITEBACK_CACHE option

As of protocol 7.23, fuse file systems can specify their cache behavior on a
per-mountpoint basis. If they set FUSE_WRITEBACK_CACHE in
fuse_init_out.flags, then they'll get the writeback cache. If not, then
they'll get the writethrough cache. If they set FOPEN_DIRECT_IO in every
FUSE_OPEN response, then they'll get no cache at all.

The old vfs.fusefs.data_cache_mode sysctl is ignored for servers that use
protocol 7.23 or later. However, it's retained for older servers,
especially for those running in jails that lack access to the new protocol.

This commit also fixes two other minor test bugs:
* WriteCluster:SetUp was using an uninitialized variable.
* Read.direct_io_pread wasn't verifying that the cache was actually
bypassed.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: fix the tests for non-default values of MAXPHYS

Sponsored by: The FreeBSD Foundation

fusefs: fix the tests for nondefault values of vfs.maxbcachebuf

Sponsored by: The FreeBSD Foundation

fusefs: rewrite vop_getpages and vop_putpages

Use the standard facilities for getpages and putpages instead of bespoke
implementations that don't work well with the writeback cache. This has
severa

fusefs: rewrite vop_getpages and vop_putpages

Use the standard facilities for getpages and putpages instead of bespoke
implementations that don't work well with the writeback cache. This has
several corollaries:

* Change the way we handle short reads _again_. vfs_bio_getpages doesn't
provide any way to handle unexpected short reads. Plus, I found some more
lock-order problems. So now when the short read is detected we'll just
clear the vnode's attribute cache, forcing the file size to be requeried
the next time it's needed. VOP_GETPAGES doesn't have any way to indicate
a short read to the "caller", so we just bzero the rest of the page
whenever a short read happens.

* Change the way we decide when to set the FUSE_WRITE_CACHE bit. We now set
it for clustered writes even when the writeback cache is not in use.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: correctly handle short reads

A fuse server may return a short read for three reasons:

* The file is opened with FOPEN_DIRECT_IO. In this case, the short read
should be returned directly

fusefs: correctly handle short reads

A fuse server may return a short read for three reasons:

* The file is opened with FOPEN_DIRECT_IO. In this case, the short read
should be returned directly to userland. We already handled this case
correctly.

* The file was truncated server-side, and the read hit EOF. In this case,
the kernel should update the file size. Fixed in the case of VOP_READ.
Fixing this for VOP_GETPAGES is TODO.

* The file is opened in writeback mode, there are dirty buffers past what
the server thinks is the file's EOF, and the read hit what the server
thinks is the file's EOF. In this case, the client is trying to read a
hole, and should zero-fill it. We already handled this case, and I added
a test for it.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: implement VOP_BMAP

If the fuse daemon supports FUSE_BMAP, then use that for the block mapping.
Otherwise, use the same technique used by vop_stdbmap. Report large values
for runp and runb i

fusefs: implement VOP_BMAP

If the fuse daemon supports FUSE_BMAP, then use that for the block mapping.
Otherwise, use the same technique used by vop_stdbmap. Report large values
for runp and runb in order to maximize read clustering and minimize upcalls,
even if we don't know the true layout.

The major result of this change is that sequential reads to FUSE files will
now usually happen 128KB at a time instead of 64KB.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: use cluster_read for more readahead

fusefs will now use cluster_read. This allows readahead of more than one
cache block. However, it won't yet actually cluster the reads because that
requ

fusefs: use cluster_read for more readahead

fusefs will now use cluster_read. This allows readahead of more than one
cache block. However, it won't yet actually cluster the reads because that
requires VOP_BMAP, which fusefs does not yet implement.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: implement non-clustered readahead

fusefs will now read ahead at most one cache block at a time (usually 64
KB). Clustered reads are still TODO. Individual file systems may disable
read ahe

fusefs: implement non-clustered readahead

fusefs will now read ahead at most one cache block at a time (usually 64
KB). Clustered reads are still TODO. Individual file systems may disable
read ahead by setting fuse_init_out.max_readahead=0 during initialization.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: rename the ReadCacheable.default_readahead test

The test didn't actually have anything to do with readahead. Rename it to
"ReadCacheable.cache_block"

Sponsored by: The FreeBSD Foundation

fusefs: make the tests more cplusplusy

* Prefer std::unique_ptr to raw pointers
* Prefer pass-by-reference to pass-by-pointer
* Prefer static_cast to C-style cast, unless it's too much typing

Repor

fusefs: make the tests more cplusplusy

* Prefer std::unique_ptr to raw pointers
* Prefer pass-by-reference to pass-by-pointer
* Prefer static_cast to C-style cast, unless it's too much typing

Reported by: ngie
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: more build fixes

* Fix printf format strings on 32-bit OSes
* Fix -Wclass-memaccess violation on GCC-8 caused by using memset on an object
of non-trivial type.
* Fix memory leak in MockFS:

fusefs: more build fixes

* Fix printf format strings on 32-bit OSes
* Fix -Wclass-memaccess violation on GCC-8 caused by using memset on an object
of non-trivial type.
* Fix memory leak in MockFS::init
* Fix -Wcast-align error on i386 in expect_readdir
* Fix some heterogenous comparison errors on 32-bit OSes.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: implement FUSE_ASYNC_READ

If a daemon sets the FUSE_ASYNC_READ flag during initialization, then the
client is allowed to issue multiple concurrent reads for the same file
handle. Otherwise

fusefs: implement FUSE_ASYNC_READ

If a daemon sets the FUSE_ASYNC_READ flag during initialization, then the
client is allowed to issue multiple concurrent reads for the same file
handle. Otherwise concurrent reads are not allowed. This commit implements
it. Previously we unconditionally disallowed concurrent reads.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: Upgrade FUSE protocol to version 7.9.

This commit upgrades the FUSE API to protocol 7.9 and adds unit tests for
backwards compatibility with servers built for version 7.8. It doesn't
implem

fusefs: Upgrade FUSE protocol to version 7.9.

This commit upgrades the FUSE API to protocol 7.9 and adds unit tests for
backwards compatibility with servers built for version 7.8. It doesn't
implement any of 7.9's new features yet.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: shorten and consolidate sleeps

Some fusefs tests must sleep because they deliberately trigger a race, or
because they're testing the cache timeout functionality. Consolidate the
sleep inter

fusefs: shorten and consolidate sleeps

Some fusefs tests must sleep because they deliberately trigger a race, or
because they're testing the cache timeout functionality. Consolidate the
sleep interval in a single place so it will be easy to adjust. Shorten it
from either 500ms or 250ms to 100ms. From experiment I find that 10ms works
every time, so 100ms should be fairly safe.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: skip the Read.keep_cache test when cache is disabled

This should've been part of r345892

PR: 236560
Sponsored by: The FreeBSD Foundation

fusefs: cache file attributes

FUSE_LOOKUP, FUSE_GETATTR, FUSE_SETATTR, FUSE_MKDIR, FUSE_LINK,
FUSE_SYMLINK, FUSE_MKNOD, and FUSE_CREATE all return file attributes with a
cache validity period. fuse

fusefs: cache file attributes

FUSE_LOOKUP, FUSE_GETATTR, FUSE_SETATTR, FUSE_MKDIR, FUSE_LINK,
FUSE_SYMLINK, FUSE_MKNOD, and FUSE_CREATE all return file attributes with a
cache validity period. fusefs will now cache the attributes, if the server
returns a non-zero cache validity period.

This change does _not_ implement finite attr cache timeouts. That will
follow as part of PR 235773.

PR: 235775
Reported by: cem
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: properly handle FOPEN_KEEP_CACHE

If a fuse file system returne FOPEN_KEEP_CACHE in the open or create
response, then the client is supposed to _not_ clear its caches for that
file. I don't

fusefs: properly handle FOPEN_KEEP_CACHE

If a fuse file system returne FOPEN_KEEP_CACHE in the open or create
response, then the client is supposed to _not_ clear its caches for that
file. I don't know why clearing the caches would be the default given that
there's a separate flag to bypass the cache altogether, but that's the way
it is. fusefs(5) will now honor this flag.

Our behavior is slightly different than Linux's because we reuse file
handles. That means that open(2) wont't clear the cache if there's a
reusable file handle, even if the file server wouldn't have sent
FOPEN_KEEP_CACHE had we opened a new file handle like Linux does.

PR: 236560
Sponsored by: The FreeBSD Foundation

show more ...

fusefs: fix more tests when data caching is disabled

readahead is also disallowed when data_cache_mode=0. This should've been
part of r345720.

Sponsored by: The FreeBSD Foundation

fusefs: fix tests when data caching is disabled

VOP_GETPAGES is disabled when vfs.fusefs.data_cache_mode=0, causing mmap to
return success but accessing the mapped memory will subsequently segfault.

fusefs: fix tests when data caching is disabled

VOP_GETPAGES is disabled when vfs.fusefs.data_cache_mode=0, causing mmap to
return success but accessing the mapped memory will subsequently segfault.

Sponsored by: The FreeBSD Foundation

show more ...

fusefs: adapt the tests to the fuse => fusefs rename

Sponsored by: The FreeBSD Foundation