sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

opencrypto: Add a routine to copy a crypto buffer cursor

This was useful in converting armv8crypto to use buffer cursors. There
are some cases where one wants to make two passes over data, and this

opencrypto: Add a routine to copy a crypto buffer cursor

This was useful in converting armv8crypto to use buffer cursors. There
are some cases where one wants to make two passes over data, and this
provides a way to "reset" a cursor.

Reviewed by: jhb
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D28949

show more ...

opencrypto/cryptodev.h: Add includes to make more self-contained.

Reviewed by: markj, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34021

crypto: Add support for the XChaCha20-Poly1305 AEAD cipher.

This cipher is a wrapper around the ChaCha20-Poly1305 AEAD cipher
which accepts a larger nonce. Part of the nonce is used along with
the

crypto: Add support for the XChaCha20-Poly1305 AEAD cipher.

This cipher is a wrapper around the ChaCha20-Poly1305 AEAD cipher
which accepts a larger nonce. Part of the nonce is used along with
the key as an input to HChaCha20 to generate a derived key used for
ChaCha20-Poly1305.

This cipher is used by WireGuard.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33523

show more ...

OCF: Add crypto_clonereq().

This function clones an existing crypto request, but associates the
new request with a specified session. The intended use case is for
drivers to be able to fall back to

OCF: Add crypto_clonereq().

This function clones an existing crypto request, but associates the
new request with a specified session. The intended use case is for
drivers to be able to fall back to software by cloning a request and
dispatch it to an internally allocated software session.

Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33607

show more ...

cryptodev.h: Drop 'extern' from function prototypes.

Sponsored by: Chelsio Communications

crypto: Define POLY1305_BLOCK_LEN constant.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33485

crypto: Support multiple nonce lengths for AES-CCM.

Permit nonces of lengths 7 through 13 in the OCF framework and the
cryptosoft driver. A helper function (ccm_max_payload_length) can be
used in O

crypto: Support multiple nonce lengths for AES-CCM.

Permit nonces of lengths 7 through 13 in the OCF framework and the
cryptosoft driver. A helper function (ccm_max_payload_length) can be
used in OCF drivers to reject CCM requests which are too large for the
specified nonce length.

Reviewed by: sef
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32111

show more ...

cryptodev: Permit explicit IV/nonce and MAC/tag lengths.

Add 'ivlen' and 'maclen' fields to the structure used for CIOGSESSION2
to specify the explicit IV/nonce and MAC/tag lengths for crypto
sessio

cryptodev: Permit explicit IV/nonce and MAC/tag lengths.

Add 'ivlen' and 'maclen' fields to the structure used for CIOGSESSION2
to specify the explicit IV/nonce and MAC/tag lengths for crypto
sessions. If these fields are zero, the default lengths are used.

This permits selecting an alternate nonce length for AEAD ciphers such
as AES-CCM which support multiple nonce leengths. It also supports
truncated MACs as input to AEAD or ETA requests.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32107

show more ...

crypto: Constify all transform descriptors

No functional change intended.

Reviewed by: ae, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.

crypto: Constify all transform descriptors

No functional change intended.

Reviewed by: ae, jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31196

show more ...

crypto: Remove now-unused crypto_cursor_seg{base,len}.

Callers should use crypto_cursor_segment() instead.

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.or

crypto: Remove now-unused crypto_cursor_seg{base,len}.

Callers should use crypto_cursor_segment() instead.

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30448

show more ...

crypto: Add crypto_cursor_segment() to fetch both base and length.

This function combines crypto_cursor_segbase() and
crypto_cursor_seglen() into a single function. This is mostly
beneficial in the

crypto: Add crypto_cursor_segment() to fetch both base and length.

This function combines crypto_cursor_segbase() and
crypto_cursor_seglen() into a single function. This is mostly
beneficial in the unmapped mbuf case where back to back calls of these
two functions have to iterate over the sub-components of unmapped
mbufs twice.

Bump __FreeBSD_version for crypto drivers in ports.

Suggested by: markj
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30445

show more ...

crypto: Add a new type of crypto buffer for a single mbuf.

This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
U

crypto: Add a new type of crypto buffer for a single mbuf.

This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
Using the existing CRYPTO_BUF_MBUF would result in
bus_dmamap_load_crp() walking additional mbufs in the socket buffer
that are not relevant, but generating a S/G list that potentially
exceeds the limit of the tag (while also wasting CPU cycles).

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30136

show more ...

OCF: Remove support for asymmetric cryptographic operations.

There haven't been any non-obscure drivers that supported this
functionality and it has been impossible to test to ensure that it
still w

OCF: Remove support for asymmetric cryptographic operations.

There haven't been any non-obscure drivers that supported this
functionality and it has been impossible to test to ensure that it
still works. The only known consumer of this interface was the engine
in OpenSSL < 1.1. Modern OpenSSL versions do not include support for
this interface as it was not well-documented.

Reviewed by: cem
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29736

show more ...

Add an OCF algorithm for ChaCha20-Poly1305 AEAD.

Note that this algorithm implements the mode defined in RFC 8439.

Reviewed by: cem
Sponsored by: Netflix
Differential Revision: https://reviews.free

Add an OCF algorithm for ChaCha20-Poly1305 AEAD.

Note that this algorithm implements the mode defined in RFC 8439.

Reviewed by: cem
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27836

show more ...

opencrypto: Introduce crypto_dispatch_async()

Currently, OpenCrypto consumers can request asynchronous dispatch by
setting a flag in the cryptop. (Currently only IPSec may do this.) I
think this

opencrypto: Introduce crypto_dispatch_async()

Currently, OpenCrypto consumers can request asynchronous dispatch by
setting a flag in the cryptop. (Currently only IPSec may do this.) I
think this is a bit confusing: we (conditionally) set cryptop flags to
request async dispatch, and then crypto_dispatch() immediately examines
those flags to see if the consumer wants async dispatch. The flag names
are also confusing since they don't specify what "async" applies to:
dispatch or completion.

Add a new KPI, crypto_dispatch_async(), rather than encoding the
requested dispatch type in each cryptop. crypto_dispatch_async() falls
back to crypto_dispatch() if the session's driver provides asynchronous
dispatch. Get rid of CRYPTOP_ASYNC() and CRYPTOP_ASYNC_KEEPORDER().

Similarly, add crypto_dispatch_batch() to request processing of a tailq
of cryptops, rather than encoding the scheduling policy using cryptop
flags. Convert GELI, the only user of this interface (disabled by
default) to use the new interface.

Add CRYPTO_SESS_SYNC(), which can be used by consumers to determine
whether crypto requests will be dispatched synchronously. This is just
a helper macro. Use it instead of looking at cap flags directly.

Fix style in crypto_done(). Also get rid of CRYPTO_RETW_EMPTY() and
just check the relevant queues directly. This could result in some
unnecessary wakeups but I think it's very uncommon to be using more than
one queue per worker in a given workload, so checking all three queues
is a waste of cycles.

Reviewed by: jhb
Sponsored by: Ampere Computing
Submitted by: Klara, Inc.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28194

show more ...

Remove the cloned file descriptors for /dev/crypto.

Crypto file descriptors were added in the original OCF import as a way
to provide per-open data (specifically the list of symmetric
sessions). Ho

Remove the cloned file descriptors for /dev/crypto.

Crypto file descriptors were added in the original OCF import as a way
to provide per-open data (specifically the list of symmetric
sessions). However, this gives a bit of a confusing API where one has
to open /dev/crypto and then invoke an ioctl to obtain a second file
descriptor. This also does not match the API used with /dev/crypto on
other BSDs or with Linux's /dev/crypto driver.

Character devices have gained support for per-open data via cdevpriv
since OCF was imported, so use cdevpriv to simplify the userland API
by permitting ioctls directly on /dev/crypto descriptors.

To provide backwards compatibility, CRIOGET now opens another
/dev/crypto descriptor via kern_openat() rather than dup'ing the
existing file descriptor. This preserves prior semantics in case
CRIOGET is invoked multiple times on a single file descriptor.

Reviewed by: markj
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27302

show more ...

Use void * in place of caddr_t.

Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27065

Consistently use C99 fixed-width types in the in-kernel crypto code.

Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27061

opencrypto: Annotate hmac_init_(i|o)pad() to make auth_hash const

This makes them friendlier to drivers that try to use const pointers
whenever possible in their internal structures.

Reviewed by: j

opencrypto: Annotate hmac_init_(i|o)pad() to make auth_hash const

This makes them friendlier to drivers that try to use const pointers
whenever possible in their internal structures.

Reviewed by: jhb
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision: https://reviews.freebsd.org/D26901

show more ...

Prepare crypto framework for IPsec ESN support

This permits requests (netipsec ESP and AH protocol) to provide the
IPsec ESN (Extended Sequence Numbers) in a separate buffer.

As with separate outpu

Prepare crypto framework for IPsec ESN support

This permits requests (netipsec ESP and AH protocol) to provide the
IPsec ESN (Extended Sequence Numbers) in a separate buffer.

As with separate output buffer and separate AAD buffer not all drivers
support this feature. Consumer must request use of this feature via new
session flag.

Submitted by: Grzegorz Jaszczyk <jaz@semihalf.com>
Patryk Duda <pdk@semihalf.com>
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D24838
Obtained from: Semihalf
Sponsored by: Stormshield

show more ...

crypto(9): add CRYPTO_BUF_VMPAGE

crypto(9) functions can now be used on buffers composed of an array of
vm_page_t structures, such as those stored in an unmapped struct bio. It
requires the running

crypto(9): add CRYPTO_BUF_VMPAGE

crypto(9) functions can now be used on buffers composed of an array of
vm_page_t structures, such as those stored in an unmapped struct bio. It
requires the running to kernel to support the direct memory map, so not all
architectures can use it.

Reviewed by: markj, kib, jhb, mjg, mat, bcr (manpages)
MFC after: 1 week
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D25671

show more ...

Add crypto_initreq() and crypto_destroyreq().

These routines are similar to crypto_getreq() and crypto_freereq() but
operate on caller-supplied storage instead of allocating crypto
requests from a U

Add crypto_initreq() and crypto_destroyreq().

These routines are similar to crypto_getreq() and crypto_freereq() but
operate on caller-supplied storage instead of allocating crypto
requests from a UMA zone.

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D25691

show more ...

Convert cryptostats to a counter_u64 array.

The global counters were not SMP-friendly. Use per-CPU counters
instead.

Reviewed by: jhb
Sponsored by: Rubicon Communications, LLC (Netgate)
Differenti

Convert cryptostats to a counter_u64 array.

The global counters were not SMP-friendly. Use per-CPU counters
instead.

Reviewed by: jhb
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision: https://reviews.freebsd.org/D25466

show more ...

Remove CRYPTO_TIMING.

It was added a very long time ago. It is single-threaded, so only
really useful for basic measurements, and in the meantime we've gotten
some more sophisticated profiling tool

Remove CRYPTO_TIMING.

It was added a very long time ago. It is single-threaded, so only
really useful for basic measurements, and in the meantime we've gotten
some more sophisticated profiling tools.

Reviewed by: cem, delphij, jhb
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision: https://reviews.freebsd.org/D25464

show more ...