sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

ioat: Add Ice Lake ID.

MFC after: 2 weeks

ioat: Remove unused devclass argument to DRIVER_MODULE.

ioat: plug set-but-not-unused vars

Sponsored by: Rubicon Communications, LLC ("Netgate")

ioat(4): Remove Giant from ioat_test enable/disable.

MFC after: 1 month

Mark some sysctls as CTLFLAG_MPSAFE.

MFC after: 2 weeks

Remove free_domain() and uma_zfree_domain().

These functions were introduced before UMA started ensuring that freed
memory gets placed in domain-local caches. They no longer serve any
purpose since

Remove free_domain() and uma_zfree_domain().

These functions were introduced before UMA started ensuring that freed
memory gets placed in domain-local caches. They no longer serve any
purpose since UMA now provides their functionality by default. Remove
them to simplyify the kernel memory allocator interfaces a bit.

Reviewed by: cem, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25937

show more ...

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

Don't spin on cleanup_lock if we are not interrupt.

If somebody else holds that lock, it will likely do the work for us.
If it won't, then we return here later and retry.

Under heavy load it allows

Don't spin on cleanup_lock if we are not interrupt.

If somebody else holds that lock, it will likely do the work for us.
If it won't, then we return here later and retry.

Under heavy load it allows to avoid lock congestion between interrupt and
polling threads.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

show more ...

Add ioat_get_domain() to ioat(4) KPI.

This allows NUMA-aware consumers to reduce inter-domain traffic.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

Initialize *comp_update with valid value.

I've noticed that sometimes with enabled DMAR initial write from device
to this address is somehow getting delayed, triggering assertion due to
zero default

Initialize *comp_update with valid value.

I've noticed that sometimes with enabled DMAR initial write from device
to this address is somehow getting delayed, triggering assertion due to
zero default being invalid.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

show more ...

Cleanup address range checks in ioat(4).

- Deduce allowed address range for bus_dma(9) from the hardware version.
Different versions (CPU generations) have different documented limits.
- Remove di

Cleanup address range checks in ioat(4).

- Deduce allowed address range for bus_dma(9) from the hardware version.
Different versions (CPU generations) have different documented limits.
- Remove difference between address ranges for src/dst and crc. At least
docs for few recent generations of CPUs do not mention anything like that,
while older are already limited with above limits.
- Remove address assertions from arguments. While I do not think the
addresses out of allowed ranges should realistically happen there due to
the platforms physical address limitations, there is now bus_dma(9) to
make sure of that, preferably via IOMMU.
- Since crc now has the same address range as src/dst, remove crc_dmamap,
reusing dst2_dmamap instead.

Discussed with: cem
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

show more ...

Pass more reasonable WAIT flags to bus_dma(9) calls.

MFC after: 2 weeks

Add CTLFLAG_STATS to the dev.ioat.N.stats sysctl OIDs

Refer to r353111.

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Replace argument checks with assertions.

Those functions are used by kernel, and we can't check all possible argument
errors in production kernel. Plus according to docs many of those errors
are ch

Replace argument checks with assertions.

Those functions are used by kernel, and we can't check all possible argument
errors in production kernel. Plus according to docs many of those errors
are checked by hardware. Assertions should just help with code debugging.

MFC after: 2 weeks

show more ...

Improve ioat(4) NUMA-awareness.

Allocate ioat->ring memory from the device domain.
Schedule ioat->poll_timer to the first CPU of the device domain.

According to pcm-numa tool from intel-pcm port, t

Improve ioat(4) NUMA-awareness.

Allocate ioat->ring memory from the device domain.
Schedule ioat->poll_timer to the first CPU of the device domain.

According to pcm-numa tool from intel-pcm port, this reduces number of
remote DRAM accesses while copying data by 75%. And unless it is a noise,
I've noticed some speed improvement when copying data to other domain.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

show more ...

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces hea

Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.

show more ...

ioat(4) should use bus_dma(9) for the operation source and destination
addresses

Reviewed by: cem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D19725

Fix few issues in ioat(4) driver.

- Do not explicitly count active descriptors. It allows hardware reset
to happen while device is still referenced, plus simplifies locking.
- Do not stop/start c

Fix few issues in ioat(4) driver.

- Do not explicitly count active descriptors. It allows hardware reset
to happen while device is still referenced, plus simplifies locking.
- Do not stop/start callout each time the queue becomes empty. Let it
run to completion and rearm if needed, that is much cheaper then to touch
it every time, plus also simplifies locking.
- Decouple submit and cleanup locks, making driver reentrant.
- Avoid memory mapped status register read on every interrupt.
- Improve locking during device attach/detach.
- Remove some no longer used variables.

Reviewed by: cem
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D19231

show more ...

Reapply, with minor tweaks, r338025, from the original commit:

Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INF

Reapply, with minor tweaks, r338025, from the original commit:

Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely. The 'table' parameter is now required to
have correct pointer (or array) type. Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.

Mostly done with the coccinelle 'spatch' tool:

$ cat modpnpsize0.cocci
@normaltables@
identifier b,c;
expression a,d,e;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,d,
-sizeof(d[0]),
e);

@singletons@
identifier b,c,d;
expression a;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,&d,
-sizeof(d),
1);

$ rg -l MODULE_PNP_INFO -- sys | \
xargs spatch --in-place --sp-file modpnpsize0.cocci

(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not. So I had to link gdiff into
PATH as diff to use spatch.)

Tinderbox'd (-DMAKE_JUST_KERNELS).
Approved by: re (glen)

show more ...

Back out r338035 until Warner is finished churning GSoC PNP patches

I was not aware Warner was making or planning to make forward progress in
this area and have since been informed of that.

It's ea

Back out r338035 until Warner is finished churning GSoC PNP patches

I was not aware Warner was making or planning to make forward progress in
this area and have since been informed of that.

It's easy to apply/reapply when churn dies down.

show more ...

Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely. The 'table' parameter is now required to
have

Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely. The 'table' parameter is now required to
have correct pointer (or array) type. Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.

Mostly done with the coccinelle 'spatch' tool:

$ cat modpnpsize0.cocci
@normaltables@
identifier b,c;
expression a,d,e;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,d,
-sizeof(d[0]),
e);

@singletons@
identifier b,c,d;
expression a;
declarer MODULE_PNP_INFO;
@@
MODULE_PNP_INFO(a,b,c,&d,
-sizeof(d),
1);

$ rg -l MODULE_PNP_INFO -- sys | \
xargs spatch --in-place --sp-file modpnpsize0.cocci

(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not. So I had to link gdiff into
PATH as diff to use spatch.)

Tinderbox'd (-DMAKE_JUST_KERNELS).

show more ...

Use '#' rather than some made up name for fields we want to ignore.

ioat(4): Add Skylake Xeon PCI-ID

SKX IOAT is just another 3.2 version of the CBDMA engine.

Submitted by: Deepak Veliath <deepak.veliath AT isilon.com>
Sponsored by: Dell EMC Isilon

Add PNP metadata to a few drivers

An eventual devd(8) or other component should be able to scan buses and
automatically load drivers that match device ids described in this metadata.

Reviewed by: i

Add PNP metadata to a few drivers

An eventual devd(8) or other component should be able to scan buses and
automatically load drivers that match device ids described in this metadata.

Reviewed by: imp
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12364

show more ...