sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

sys/libkern: Use C99 fixed-width integer types.

No functional change.

Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D33637

arc4random: Avoid KMSAN false positives from pre-seeding results

If code calls arc4random(), and our RNG is not yet seeded and
random_bypass_before_seeding is true, we'll compute a key using the
SHA

arc4random: Avoid KMSAN false positives from pre-seeding results

If code calls arc4random(), and our RNG is not yet seeded and
random_bypass_before_seeding is true, we'll compute a key using the
SHA256 hash of some hopefully hard-to-predict data, including the
contents of an uninitialized stack buffer (which is also the output
buffer).

When KMSAN is enabled, this use of uninitialized state propagtes through
to the arc4random() output, resulting in false positives. To address
this, lie to KMSAN and explicitly mark the buffer as initialized.

Reviewed by: cem (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31510

show more ...

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

S

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

Sponsored by: The FreeBSD Foundation

show more ...

arc4random(9): Integrate with RANDOM_FENESTRASX push-reseed

There is no functional change for the existing Fortuna random(4)
implementation, which remains the default in GENERIC.

In the FenestrasX

arc4random(9): Integrate with RANDOM_FENESTRASX push-reseed

There is no functional change for the existing Fortuna random(4)
implementation, which remains the default in GENERIC.

In the FenestrasX model, when the root CSPRNG is reseeded from pools due to
an (infrequent) timer, child CSPRNGs can cheaply detect this condition and
reseed. To do so, they just need to track an additional 64-bit value in the
associated state, and compare it against the root seed version (generation)
on random reads.

This revision integrates arc4random(9) into that model without substantially
changing the design or implementation of arc4random(9). The motivation is
that arc4random(9) is immediately reseeded when the backing random(4)
implementation has additional entropy. This is arguably most important
during boot, when fenestrasX is reseeding at 1, 3, 9, 27, etc., second
intervals. Today, arc4random(9) has a hardcoded 300 second reseed window.
Without this mechanism, if arc4random(9) gets weak entropy during initial
seed (and arc4random(9) is used early in boot, so this is quite possible),
it may continue to emit poorly seeded output for 5 minutes. The FenestrasX
push-reseed scheme corrects consumers, like arc4random(9), as soon as
possible.

Reviewed by: markm
Approved by: csprng (markm)
Differential Revision: https://reviews.freebsd.org/D22838

show more ...

random(4): depessimize arc4random

- __predict_false reseeding on entry as it is almost never true.
- don't blindly atomic_cmpset as on x86 it ends up dirtying the cacheline.
it almost ever succeeds

random(4): depessimize arc4random

- __predict_false reseeding on entry as it is almost never true.
- don't blindly atomic_cmpset as on x86 it ends up dirtying the cacheline.
it almost ever succeeds per above
- fetch the timestamp prior to getting the cpu number

Reviewed by: cem
Approved by: secteam (delphij)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20242

show more ...

random(4): Restore availability tradeoff prior to r346250

As discussed in that commit message, it is a dangerous default. But the
safe default causes enough pain on a variety of platforms that for

random(4): Restore availability tradeoff prior to r346250

As discussed in that commit message, it is a dangerous default. But the
safe default causes enough pain on a variety of platforms that for now,
restore the prior default.

Some of this is self-induced pain we should/could do better about; for
example, programmatic CI systems and VM managers should introduce entropy
from the host for individual VM instances. This is considered a future work
item.

On modern x86 and Power9 systems, this may be wholly unnecessary after
D19928 lands (even in the non-ideal case where early /boot/entropy is
unavailable), because they have fast hardware random sources available early
in boot. But D19928 is not yet landed and we have a host of architectures
which do not provide fast random sources.

This change adds several tunables and diagnostic sysctls, documented
thoroughly in UPDATING and sys/dev/random/random_infra.c.

PR: 230875 (reopens)
Reported by: adrian, jhb, imp, and probably others
Reviewed by: delphij, imp (earlier version), markm (earlier version)
Discussed with: adrian
Approved by: secteam(delphij)
Relnotes: yeah
Security: related
Differential Revision: https://reviews.freebsd.org/D19944

show more ...

random(4): Block read_random(9) on initial seeding

read_random() is/was used, mostly without error checking, in a lot of
very sensitive places in the kernel -- including seeding the widely used
arc4

random(4): Block read_random(9) on initial seeding

read_random() is/was used, mostly without error checking, in a lot of
very sensitive places in the kernel -- including seeding the widely used
arc4random(9).

Most uses, especially arc4random(9), should block until the device is seeded
rather than proceeding with a bogus or empty seed. I did not spy any
obvious kernel consumers where blocking would be inappropriate (in the
sense that lack of entropy would be ok -- I did not investigate locking
angle thoroughly). In many instances, arc4random_buf(9) or that family
of APIs would be more appropriate anyway; that work was done in r345865.

A minor cleanup was made to the implementation of the READ_RANDOM function:
instead of using a variable-length array on the stack to temporarily store
all full random blocks sufficient to satisfy the requested 'len', only store
a single block on the stack. This has some benefit in terms of reducing
stack usage, reducing memcpy overhead and reducing devrandom output leakage
via the stack. Additionally, the stack block is now safely zeroed if it was
used.

One caveat of this change is that the kern.arandom sysctl no longer returns
zero bytes immediately if the random device is not seeded. This means that
FreeBSD-specific userspace applications which attempted to handle an
unseeded random device may be broken by this change. If such behavior is
needed, it can be replaced by the more portable getrandom(2) GRND_NONBLOCK
option.

On any typical FreeBSD system, entropy is persisted on read/write media and
used to seed the random device very early in boot, and blocking is never a
problem.

This change primarily impacts the behavior of /dev/random on embedded
systems with read-only media that do not configure "nodevice random". We
toggle the default from 'charge on blindly with no entropy' to 'block
indefinitely.' This default is safer, but may cause frustration. Embedded
system designers using FreeBSD have several options. The most obvious is to
plan to have a small writable NVRAM or NAND to persist entropy, like larger
systems. Early entropy can be fed from any loader, or by writing directly
to /dev/random during boot. Some embedded SoCs now provide a fast hardware
entropy source; this would also work for quickly seeding Fortuna. A 3rd
option would be creating an embedded-specific, more simplistic random
module, like that designed by DJB in [1] (this design still requires a small
rewritable media for forward secrecy). Finally, the least preferred option
might be "nodevice random", although I plan to remove this in a subsequent
revision.

To help developers emulate the behavior of these embedded systems on
ordinary workstations, the tunable kern.random.block_seeded_status was
added. When set to 1, it blocks the random device.

I attempted to document this change in random.4 and random.9 and ran into a
bunch of out-of-date or irrelevant or inaccurate content and ended up
rototilling those documents more than I intended to. Sorry. I think
they're in a better state now.

PR: 230875
Reviewed by: delphij, markm (earlier version)
Approved by: secteam(delphij), devrandom(markm)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D19744

show more ...

Replace the RC4 algorithm for generating in-kernel secure random
numbers with Chacha20. Keep the API, though, as that is what the
other *BSD's have done.

Use the boot-time entropy stash (if present)

Replace the RC4 algorithm for generating in-kernel secure random
numbers with Chacha20. Keep the API, though, as that is what the
other *BSD's have done.

Use the boot-time entropy stash (if present) to bootstrap the
in-kernel entropy source.

Reviewed by: delphij,rwatson
Approved by: so(delphij)
MFC after: 2 months
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D10048

show more ...

Discard first 3072 bytes of RC4 keystream, this is a bandaid
that allows us to work on switching to a more modern PRNG.

Submitted by: Steven Chamberlain <steven pyro eu org>
Approved by: so

Update r309143 to prevent false sharing.

Reported by: mjg
Approved by: so
MFC after: 1 month

In a dual processor system (2*6 cores) during IPSec throughput tests,
we see a lot of contention on the arc4 lock, used to generate the IV
of the ESP output packets.

The idea of this patch is to spl

In a dual processor system (2*6 cores) during IPSec throughput tests,
we see a lot of contention on the arc4 lock, used to generate the IV
of the ESP output packets.

The idea of this patch is to split this mutex in order to reduce the
contention on this lock.

Reviewed by: delphij, markm, ache
Approved by: so
Obtained from: emeric.poupon@stormshield.eu
MFC after: 1 month
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D8130

show more ...

Huge cleanup of random(4) code.

* GENERAL
- Update copyright.
- Make kernel options for RANDOM_YARROW and RANDOM_DUMMY. Set
neither to ON, which means we want Fortuna
- If there is no 'device rand

Huge cleanup of random(4) code.

* GENERAL
- Update copyright.
- Make kernel options for RANDOM_YARROW and RANDOM_DUMMY. Set
neither to ON, which means we want Fortuna
- If there is no 'device random' in the kernel, there will be NO
random(4) device in the kernel, and the KERN_ARND sysctl will
return nothing. With RANDOM_DUMMY there will be a random(4) that
always blocks.
- Repair kern.arandom (KERN_ARND sysctl). The old version went
through arc4random(9) and was a bit weird.
- Adjust arc4random stirring a bit - the existing code looks a little
suspect.
- Fix the nasty pre- and post-read overloading by providing explictit
functions to do these tasks.
- Redo read_random(9) so as to duplicate random(4)'s read internals.
This makes it a first-class citizen rather than a hack.
- Move stuff out of locked regions when it does not need to be
there.
- Trim RANDOM_DEBUG printfs. Some are excess to requirement, some
behind boot verbose.
- Use SYSINIT to sequence the startup.
- Fix init/deinit sysctl stuff.
- Make relevant sysctls also tunables.
- Add different harvesting "styles" to allow for different requirements
(direct, queue, fast).
- Add harvesting of FFS atime events. This needs to be checked for
weighing down the FS code.
- Add harvesting of slab allocator events. This needs to be checked for
weighing down the allocator code.
- Fix the random(9) manpage.
- Loadable modules are not present for now. These will be re-engineered
when the dust settles.
- Use macros for locks.
- Fix comments.

* src/share/man/...
- Update the man pages.

* src/etc/...
- The startup/shutdown work is done in D2924.

* src/UPDATING
- Add UPDATING announcement.

* src/sys/dev/random/build.sh
- Add copyright.
- Add libz for unit tests.

* src/sys/dev/random/dummy.c
- Remove; no longer needed. Functionality incorporated into randomdev.*.

* live_entropy_sources.c live_entropy_sources.h
- Remove; content moved.
- move content to randomdev.[ch] and optimise.

* src/sys/dev/random/random_adaptors.c src/sys/dev/random/random_adaptors.h
- Remove; plugability is no longer used. Compile-time algorithm
selection is the way to go.

* src/sys/dev/random/random_harvestq.c src/sys/dev/random/random_harvestq.h
- Add early (re)boot-time randomness caching.

* src/sys/dev/random/randomdev_soft.c src/sys/dev/random/randomdev_soft.h
- Remove; no longer needed.

* src/sys/dev/random/uint128.h
- Provide a fake uint128_t; if a real one ever arrived, we can use
that instead. All that is needed here is N=0, N++, N==0, and some
localised trickery is used to manufacture a 128-bit 0ULLL.

* src/sys/dev/random/unit_test.c src/sys/dev/random/unit_test.h
- Improve unit tests; previously the testing human needed clairvoyance;
now the test will do a basic check of compressibility. Clairvoyant
talent is still a good idea.
- This is still a long way off a proper unit test.

* src/sys/dev/random/fortuna.c src/sys/dev/random/fortuna.h
- Improve messy union to just uint128_t.
- Remove unneeded 'static struct fortuna_start_cache'.
- Tighten up up arithmetic.
- Provide a method to allow eternal junk to be introduced; harden
it against blatant by compress/hashing.
- Assert that locks are held correctly.
- Fix the nasty pre- and post-read overloading by providing explictit
functions to do these tasks.
- Turn into self-sufficient module (no longer requires randomdev_soft.[ch])

* src/sys/dev/random/yarrow.c src/sys/dev/random/yarrow.h
- Improve messy union to just uint128_t.
- Remove unneeded 'staic struct start_cache'.
- Tighten up up arithmetic.
- Provide a method to allow eternal junk to be introduced; harden
it against blatant by compress/hashing.
- Assert that locks are held correctly.
- Fix the nasty pre- and post-read overloading by providing explictit
functions to do these tasks.
- Turn into self-sufficient module (no longer requires randomdev_soft.[ch])
- Fix some magic numbers elsewhere used as FAST and SLOW.

Differential Revision: https://reviews.freebsd.org/D2025
Reviewed by: vsevolod,delphij,rwatson,trasz,jmg
Approved by: so (delphij)

show more ...

Attempt to mitigate poor initialization of arc4 by one-shot
reinitialization from yarrow right after good entropy is harvested.

Approved by: secteam (delphij)
MFC after: 1 week

Return arc4_i = arc4_j = 0; line from previous backing out since
just found it in OpenBSD, see their sys/crypto/arc4.c, function
rc4_keysetup, line ctx->x = ctx->y = 0;

Obtained from: OpenBSD

Per rwatson's request:
"If you don't get a review within a day or two, I would firmly recommend
backing out the changes"

back out all my changes as unreviewed by secteam@ yet.

1) Initialize arc4_i and arc4_j to 0 after key mixing as recommended in
draft-kaukonen-cipher-arcfour-03.txt (3.1.5)

2) Drop first 768 bytes as standard RC4-drop(768)

Lock down arc4random so it can be safely called w/o Giant.

Minor code reorganization was required, but the only functional
change was that the first 1024 bytes of output are thrown out
after each re

Lock down arc4random so it can be safely called w/o Giant.

Minor code reorganization was required, but the only functional
change was that the first 1024 bytes of output are thrown out
after each reseed, rather than just the initial seed.

show more ...

Use __FBSDID().

Slight overhaul of arc4random() and friends.

One bug fixed: Use getmicrouptime() to trigger reseeds so that we
cannot be tricked by a clock being stepped backwards.

Express parameters in natural u

Slight overhaul of arc4random() and friends.

One bug fixed: Use getmicrouptime() to trigger reseeds so that we
cannot be tricked by a clock being stepped backwards.

Express parameters in natural units and with natural names.

Don't use struct timeval more than we need to.

Various stylistic and readability polishing.

Introduce arc4rand(void *ptr, u_int len, int reseed) function which
returns a stream of pseudo-random bytes, observing the automatic
reseed criteria as well as allowing forced reseeds.

Rewrite arc4random() in terms of arc4rand().

Sponsored by: DARPA & NAI Labs.

show more ...

Fixed some typos, verboseness and misformatting in comments.

Minor improvements to arc4random():

- Decrease reseeding interval from every 64 to every 16384 runs
to reduce entropy usage.
- Add time based reseeding. (Every 5 minutes.)
- Throw away the first

Minor improvements to arc4random():

- Decrease reseeding interval from every 64 to every 16384 runs
to reduce entropy usage.
- Add time based reseeding. (Every 5 minutes.)
- Throw away the first 256 words of output as suggested in
"Weaknesses in the Key Scheduling Algorithm of RC4."

Reviewed by: Mark Murray
MFC After: jkh says ok

show more ...

Recouple arc4random(9) to the entropy device. This works whether the
entropy device is loaded or not.

If the entropy device is NOT loaded, the the quality of perturbations
supplied by this will suff

Recouple arc4random(9) to the entropy device. This works whether the
entropy device is loaded or not.

If the entropy device is NOT loaded, the the quality of perturbations
supplied by this will suffer.

show more ...

Back out previous revision now that sys/random.h is properly fixed.