sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

xdma: Fix -Wunused-but-set-variable warnings

MFC after: 1 week

Fix a KASSERT since chained mbufs are accepted by the xdma bounce
buffer loader. m_copydata() will copy entire chain to a single buffer.

Sponsored by: DARPA, AFRL

xdma: avoid NULL deref in error case

Reported by: Dr Silvio Cesare of InfoSect
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for vari

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for various BERI peripherals modelled in
software. It accepts FreeBSD/mips64 page directories format and manages
BERI TLB.

1. https://github.com/CTSRD-CHERI/device-model

Sponsored by: DARPA, AFRL

show more ...

Fix transfers that don't use busdma or bounce buffer (e.g. software dmas).
Busdma data loaded by different function (the bug introduced in r349727).

Sponsored by: DARPA, AFRL

Negate the logic of XCHAN_CAP_NOBUFS macro and rename it to
XCHAN_CAP_BOUNCE.

The only application that uses bounce buffering for now is the Government
Furnished Equipment (GFE) P2's dma core (AXIDM

Negate the logic of XCHAN_CAP_NOBUFS macro and rename it to
XCHAN_CAP_BOUNCE.

The only application that uses bounce buffering for now is the Government
Furnished Equipment (GFE) P2's dma core (AXIDMA) with its own dedicated
cacheless bounce buffer.

Sponsored by: DARPA, AFRL

show more ...

Don't copy the data from bounce buffer back to the mbuf if channel does
not use bounce buffering.

Sponsored by: DARPA, AFRL

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces hea

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.

show more ...

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from it.
The same vmem could be used by DMA engine drivers to allocate memory for
DMA descriptors.
This is required for platforms that provide uncached memory region
reserved exclusively for DMA operations.
o Change sleepable sx(9) lock type to non-sleepable mutex(9) since
network drivers usually hold mutex during DMA operations. So we don't
take sleepable lock after non-sleepable.

Tested on U.S. Government Furnished Equipment (GFE) 64-bit RISC-V cores.

Sponsored by: DARPA, AFRL

show more ...

o Rewrite softdma_process_tx() of Altera SoftDMA engine driver
so it does not require a bounce buffer. The only need for this was
to align the buffer address. Implement unaligned access and we do

o Rewrite softdma_process_tx() of Altera SoftDMA engine driver
so it does not require a bounce buffer. The only need for this was
to align the buffer address. Implement unaligned access and we don't
need to copy data twice.
o Remove contigmalloc-based bounce buffer from xDMA code since it is
not suitable for arbitrary memory provided by platform, which is
sometimes a dedicated piece of memory that is not managed by OS at all.

Sponsored by: DARPA, AFRL

show more ...

Don't include sys/bus_dma.h directly, use machine/bus.h instead.

Sponsored by: DARPA, AFRL

Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical op

Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical operation flow in peripheral device driver is:

1. Get xDMA controller
sc->xdma_tx = xdma_ofw_get(sc->dev, "tx");

2. Allocate virtual channel
sc->xchan_tx = xdma_channel_alloc(sc->xdma_tx, caps);

3. Setup transfer status callback
xdma_setup_intr(sc->xchan_tx, my_tx_intr, sc, &sc->ih_tx);

4. Request a transfer(s)
ret = xdma_request(sc->xchan_tx, &req);

5. Free the channel
xdma_channel_free(sc->xdma_tx);

6. Free the controller
xdma_put(sc->xdma_tx);

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14971

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

xdma: Fix -Wunused-but-set-variable warnings

MFC after: 1 week

Fix a KASSERT since chained mbufs are accepted by the xdma bounce
buffer loader. m_copydata() will copy entire chain to a single buffer.

Sponsored by: DARPA, AFRL

xdma: avoid NULL deref in error case

Reported by: Dr Silvio Cesare of InfoSect
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for vari

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for various BERI peripherals modelled in
software. It accepts FreeBSD/mips64 page directories format and manages
BERI TLB.

1. https://github.com/CTSRD-CHERI/device-model

Sponsored by: DARPA, AFRL

show more ...

Fix transfers that don't use busdma or bounce buffer (e.g. software dmas).
Busdma data loaded by different function (the bug introduced in r349727).

Sponsored by: DARPA, AFRL

Negate the logic of XCHAN_CAP_NOBUFS macro and rename it to
XCHAN_CAP_BOUNCE.

The only application that uses bounce buffering for now is the Government
Furnished Equipment (GFE) P2's dma core (AXIDM

Negate the logic of XCHAN_CAP_NOBUFS macro and rename it to
XCHAN_CAP_BOUNCE.

The only application that uses bounce buffering for now is the Government
Furnished Equipment (GFE) P2's dma core (AXIDMA) with its own dedicated
cacheless bounce buffer.

Sponsored by: DARPA, AFRL

show more ...

Don't copy the data from bounce buffer back to the mbuf if channel does
not use bounce buffering.

Sponsored by: DARPA, AFRL

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces hea

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.

show more ...

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from it.
The same vmem could be used by DMA engine drivers to allocate memory for
DMA descriptors.
This is required for platforms that provide uncached memory region
reserved exclusively for DMA operations.
o Change sleepable sx(9) lock type to non-sleepable mutex(9) since
network drivers usually hold mutex during DMA operations. So we don't
take sleepable lock after non-sleepable.

Tested on U.S. Government Furnished Equipment (GFE) 64-bit RISC-V cores.

Sponsored by: DARPA, AFRL

show more ...

o Rewrite softdma_process_tx() of Altera SoftDMA engine driver
so it does not require a bounce buffer. The only need for this was
to align the buffer address. Implement unaligned access and we do

o Rewrite softdma_process_tx() of Altera SoftDMA engine driver
so it does not require a bounce buffer. The only need for this was
to align the buffer address. Implement unaligned access and we don't
need to copy data twice.
o Remove contigmalloc-based bounce buffer from xDMA code since it is
not suitable for arbitrary memory provided by platform, which is
sometimes a dedicated piece of memory that is not managed by OS at all.

Sponsored by: DARPA, AFRL

show more ...

Don't include sys/bus_dma.h directly, use machine/bus.h instead.

Sponsored by: DARPA, AFRL