sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

xdma: Inline variables with simple values used once in a KASSERT().

Enter the network epoch in the xdma interrupt handler if required
by a peripheral device driver.

Sponsored by: DARPA, AFRL

Fix xae(4) driver attachement on the Government Furnished Equipment (GFE)
riscv cores.

GFE cores come with standard DTS file that lacks standard 'dmas ='
property, which means xae(4) could not find

Fix xae(4) driver attachement on the Government Furnished Equipment (GFE)
riscv cores.

GFE cores come with standard DTS file that lacks standard 'dmas ='
property, which means xae(4) could not find a DMA controller to use.

The 'dmas' property could not be added to the DTS file because the
ethernet controller and DMA engine parts in Linux are implemented
in a single driver.

Instead of 'dmas' property the standard Xilinx 'axistream-connected'
property is provided, so fallback to use it instead.

Suggested by: James Clarke <jrtc27@jrtc27.com>
Reviewed by: James Clarke <jrtc27@jrtc27.com>
Sponsored by: DARPA, AFRL

show more ...

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for vari

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for various BERI peripherals modelled in
software. It accepts FreeBSD/mips64 page directories format and manages
BERI TLB.

1. https://github.com/CTSRD-CHERI/device-model

Sponsored by: DARPA, AFRL

show more ...

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces hea

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.

show more ...

xdma(4): Fix invalid pointer use (breaks arm.SOCFPGA build)

In xdma_handle_mem_node(), vmem_size_t and vmem_addr_t pointers were passed to
an FDT API that emits u_long values to the output parameter

xdma(4): Fix invalid pointer use (breaks arm.SOCFPGA build)

In xdma_handle_mem_node(), vmem_size_t and vmem_addr_t pointers were passed to
an FDT API that emits u_long values to the output parameter pointer. This
broke on systems with both xdma and 32-bit vmem size/addr types (SOCFPGA).

Reported by: tinderbox
Sponsored by: Dell EMC Isilon

show more ...

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from it.
The same vmem could be used by DMA engine drivers to allocate memory for
DMA descriptors.
This is required for platforms that provide uncached memory region
reserved exclusively for DMA operations.
o Change sleepable sx(9) lock type to non-sleepable mutex(9) since
network drivers usually hold mutex during DMA operations. So we don't
take sleepable lock after non-sleepable.

Tested on U.S. Government Furnished Equipment (GFE) 64-bit RISC-V cores.

Sponsored by: DARPA, AFRL

show more ...

Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical op

Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical operation flow in peripheral device driver is:

1. Get xDMA controller
sc->xdma_tx = xdma_ofw_get(sc->dev, "tx");

2. Allocate virtual channel
sc->xchan_tx = xdma_channel_alloc(sc->xdma_tx, caps);

3. Setup transfer status callback
xdma_setup_intr(sc->xchan_tx, my_tx_intr, sc, &sc->ih_tx);

4. Request a transfer(s)
ret = xdma_request(sc->xchan_tx, &req);

5. Free the channel
xdma_channel_free(sc->xdma_tx);

6. Free the controller
xdma_put(sc->xdma_tx);

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14971

show more ...

Clean up MD pollution of bus_dma.h:

--Remove special-case handling of sparc64 bus_dmamap* functions.
Replace with a more generic mechanism that allows MD busdma
implementations to generate inlin

Clean up MD pollution of bus_dma.h:

--Remove special-case handling of sparc64 bus_dmamap* functions.
Replace with a more generic mechanism that allows MD busdma
implementations to generate inline mapping functions by
defining WANT_INLINE_DMAMAP in <machine/bus_dma.h>. This
is currently useful for sparc64, x86, and arm64, which all
implement non-load dmamap operations as simple wrappers
around map objects which may be bus- or device-specific.

--Remove NULL-checked bus_dmamap macros. Implement the
equivalent NULL checks in the inlined x86 implementation.
For non-x86 platforms, these checks are a minor pessimization
as those platforms do not currently allow NULL maps. NULL
maps were originally allowed on arm64, which appears to have
been the motivation behind adding arm[64]-specific barriers
to bus_dma.h, but that support was removed in r299463.

--Simplify the internal interface used by the bus_dmamap_load*
variants and move it to bus_dma_internal.h

--Fix some drivers that directly include sys/bus_dma.h
despite the recommendations of bus_dma(9)

Reviewed by: kib (previous revision), marius
Differential Revision: https://reviews.freebsd.org/D10729

show more ...

Add parenthesis.

Noticed by: hps
Sponsored by: DARPA, AFRL

o Fix unlocking.
o Fix types for ds_addr/ds_len.

Sponsored by: DARPA, AFRL

Add xDMA -- the DMA abstraction layer, initial verison.

xDMA is a DMA framework designed to abstract the interaction
between device drivers and DMA engines.

Project wiki: https://wiki.freebsd.org/x

Add xDMA -- the DMA abstraction layer, initial verison.

xDMA is a DMA framework designed to abstract the interaction
between device drivers and DMA engines.

Project wiki: https://wiki.freebsd.org/xdma

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D8807

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

xdma: Inline variables with simple values used once in a KASSERT().

Enter the network epoch in the xdma interrupt handler if required
by a peripheral device driver.

Sponsored by: DARPA, AFRL

Fix xae(4) driver attachement on the Government Furnished Equipment (GFE)
riscv cores.

GFE cores come with standard DTS file that lacks standard 'dmas ='
property, which means xae(4) could not find

Fix xae(4) driver attachement on the Government Furnished Equipment (GFE)
riscv cores.

GFE cores come with standard DTS file that lacks standard 'dmas ='
property, which means xae(4) could not find a DMA controller to use.

The 'dmas' property could not be added to the DTS file because the
ethernet controller and DMA engine parts in Linux are implemented
in a single driver.

Instead of 'dmas' property the standard Xilinx 'axistream-connected'
property is provided, so fallback to use it instead.

Suggested by: James Clarke <jrtc27@jrtc27.com>
Reviewed by: James Clarke <jrtc27@jrtc27.com>
Sponsored by: DARPA, AFRL

show more ...

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for vari

o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for various BERI peripherals modelled in
software. It accepts FreeBSD/mips64 page directories format and manages
BERI TLB.

1. https://github.com/CTSRD-CHERI/device-model

Sponsored by: DARPA, AFRL

show more ...

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces hea

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.

show more ...

xdma(4): Fix invalid pointer use (breaks arm.SOCFPGA build)

In xdma_handle_mem_node(), vmem_size_t and vmem_addr_t pointers were passed to
an FDT API that emits u_long values to the output parameter

xdma(4): Fix invalid pointer use (breaks arm.SOCFPGA build)

In xdma_handle_mem_node(), vmem_size_t and vmem_addr_t pointers were passed to
an FDT API that emits u_long values to the output parameter pointer. This
broke on systems with both xdma and 32-bit vmem size/addr types (SOCFPGA).

Reported by: tinderbox
Sponsored by: Dell EMC Isilon

show more ...

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from

o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
"memory-region" property and use vmem(9) to allocate buffers from it.
The same vmem could be used by DMA engine drivers to allocate memory for
DMA descriptors.
This is required for platforms that provide uncached memory region
reserved exclusively for DMA operations.
o Change sleepable sx(9) lock type to non-sleepable mutex(9) since
network drivers usually hold mutex during DMA operations. So we don't
take sleepable lock after non-sleepable.

Tested on U.S. Government Furnished Equipment (GFE) 64-bit RISC-V cores.

Sponsored by: DARPA, AFRL

show more ...

Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical op

Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical operation flow in peripheral device driver is:

1. Get xDMA controller
sc->xdma_tx = xdma_ofw_get(sc->dev, "tx");

2. Allocate virtual channel
sc->xchan_tx = xdma_channel_alloc(sc->xdma_tx, caps);

3. Setup transfer status callback
xdma_setup_intr(sc->xchan_tx, my_tx_intr, sc, &sc->ih_tx);

4. Request a transfer(s)
ret = xdma_request(sc->xchan_tx, &req);

5. Free the channel
xdma_channel_free(sc->xdma_tx);

6. Free the controller
xdma_put(sc->xdma_tx);

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14971

show more ...

Clean up MD pollution of bus_dma.h:

--Remove special-case handling of sparc64 bus_dmamap* functions.
Replace with a more generic mechanism that allows MD busdma
implementations to generate inlin

Clean up MD pollution of bus_dma.h:

--Remove special-case handling of sparc64 bus_dmamap* functions.
Replace with a more generic mechanism that allows MD busdma
implementations to generate inline mapping functions by
defining WANT_INLINE_DMAMAP in <machine/bus_dma.h>. This
is currently useful for sparc64, x86, and arm64, which all
implement non-load dmamap operations as simple wrappers
around map objects which may be bus- or device-specific.

--Remove NULL-checked bus_dmamap macros. Implement the
equivalent NULL checks in the inlined x86 implementation.
For non-x86 platforms, these checks are a minor pessimization
as those platforms do not currently allow NULL maps. NULL
maps were originally allowed on arm64, which appears to have
been the motivation behind adding arm[64]-specific barriers
to bus_dma.h, but that support was removed in r299463.

--Simplify the internal interface used by the bus_dmamap_load*
variants and move it to bus_dma_internal.h

--Fix some drivers that directly include sys/bus_dma.h
despite the recommendations of bus_dma(9)

Reviewed by: kib (previous revision), marius
Differential Revision: https://reviews.freebsd.org/D10729

show more ...

Add parenthesis.

Noticed by: hps
Sponsored by: DARPA, AFRL

o Fix unlocking.
o Fix types for ds_addr/ds_len.

Sponsored by: DARPA, AFRL