sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

Intel DMAR: remove parsing of 6-level paging capability

Early versions of the VT-d spec mentioned 6-level paging support as a
possible value for the SAGAW capability, but later versions removed it
a

Intel DMAR: remove parsing of 6-level paging capability

Early versions of the VT-d spec mentioned 6-level paging support as a
possible value for the SAGAW capability, but later versions removed it
and SAGAW=0x10 is currently listed as a reserved value.

The 6-level (agaw=64) entry in sagaw_bits is furthermore problematic
with clang15 because the attempted comparison against 1ULL << 64 in
dmar_maxaddr2mgaw() causes the compiler to elide the last iteration
of the initial loop, which bypasses the subsequent logic to find the
greatest HW-supported address width. This results in 5-level paging
always being selected regardless of whether the hardware supports it,
which can result address translation failure due to invalid context-
entry programming.

Reviewed by: kib
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D39896

show more ...

dmar: Disable PMR in driver attach routine

Previously it was disabled right before translation was enabled.
This way the disable logic is still executed even when translation
is not be activated, e.

dmar: Disable PMR in driver attach routine

Previously it was disabled right before translation was enabled.
This way the disable logic is still executed even when translation
is not be activated, e.g. with hw.iommu.dma=0 tunable set.
On some platforms we need to disable PMR in order for core dump to work.
At the same time it was observed that enabling translation has
a significant impact on network performance.
With this patch PMR can be disabled, with IOMMU translation not being
turned on by appending the following to the loader.conf:

hw.dmar.enable=1
hw.dmar.pmr.disable=1
hw.dmar.dma=0

Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D34907

show more ...

dmar: Disable protected memory regions after initialization

Some BIOSes protect memory region they reside in by using DMAR to
prevent devices from doing any DMA transactions to that part of RAM.
AMI

dmar: Disable protected memory regions after initialization

Some BIOSes protect memory region they reside in by using DMAR to
prevent devices from doing any DMA transactions to that part of RAM.
AMI refers to this as "DMA Control Guarantee".
Disable the protection when address translation is enabled.
I stumbled upon this while investigation a failing coredump on a device
which has this feature enabled.

Sponsored by: Stormshield
Obtained from: Semihalf
Reviewed by: kib
Differential revision: https://reviews.freebsd.org/D32591

show more ...

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

S

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

Sponsored by: The FreeBSD Foundation

show more ...

o Don't include headers from iommu.h, include them from the header
consumers instead;
o Order includes properly.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.fr

o Don't include headers from iommu.h, include them from the header
consumers instead;
o Order includes properly.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D25878

show more ...

Rename DMAR flags:
o DMAR_DOMAIN_* -> IOMMU_DOMAIN_*
o DMAR_PGF_* -> IOMMU_PGF_*

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D25812

o Make the _hw_iommu sysctl node non-static;
o Move the dmar sysctl knobs to _hw_iommu_dmar.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D25807

o Move iommu gas prototypes, DMAR flags to iommu.h;
o Move hw.dmar sysctl node to iommu_gas.c.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D25802

Split-out the Intel GAS (Guest Address Space) management component
from Intel DMAR support, so it can be used on other IOMMU systems.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision:

Split-out the Intel GAS (Guest Address Space) management component
from Intel DMAR support, so it can be used on other IOMMU systems.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D25743

show more ...

Move the Intel DMAR busdma backend to a generic place so
it can be used on other IOMMU systems.

Reviewed by: kib
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D25720

Start splitting-out the Intel DMAR busdma backend to a generic place,
so it can be used on other IOMMU systems.

Provide MI iommu_unit, iommu_domain and iommu_ctx structs in sys/iommu.h;
use them as

Start splitting-out the Intel DMAR busdma backend to a generic place,
so it can be used on other IOMMU systems.

Provide MI iommu_unit, iommu_domain and iommu_ctx structs in sys/iommu.h;
use them as a first member of MD dmar_unit, dmar_domain and dmar_ctx.

Change the namespace in DMAR backend: use iommu_ prefix instead of dmar_.

Move some macroses and function prototypes to sys/iommu.h.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D25574

show more ...

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

Fix a few places that free a page from an object without busy held. This is
tightening constraints on busy as a precursor to lockless page lookup and
should largely be a NOP for these cases.

Review

Fix a few places that free a page from an object without busy held. This is
tightening constraints on busy as a precursor to lockless page lookup and
should largely be a NOP for these cases.

Reviewed by: alc, kib, markj
Differential Revision: https://reviews.freebsd.org/D22611

show more ...

bus_dma_dmar_set_buswide(9): KPI to indicate that the whole dmar
context should share page tables.

Practically it means that dma requests from any device on the bus are
translated according to the e

bus_dma_dmar_set_buswide(9): KPI to indicate that the whole dmar
context should share page tables.

Practically it means that dma requests from any device on the bus are
translated according to the entries loaded for the bus:0:0 device.
KPI requires that the slot and function of the device be 0:0, and that
no tags for other devices on the bus were used.

The intended use are NTBs which pass TLPs from the downstream to the
host with slot:func of the downstream originator.

Reviewed and tested by: mav
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D22434

show more ...

Correct handling of RMRR during early enumeration stages.

On some machines, DMAR contexts must be created before all devices
under the scope of the corresponding DMAR unit are enumerated.
Current co

Correct handling of RMRR during early enumeration stages.

On some machines, DMAR contexts must be created before all devices
under the scope of the corresponding DMAR unit are enumerated.
Current code has two problems with that:
- scope lookup returns NULL device_t, which causes to skip creating a
context with RMRR, which is fatal for the affected device.
- calculation of the final pci dbsf address fails if any bridge in the
scope is not yet enumerated, because code relies on pcib_get_bus().

Make creation of contexts work either with device_t, or with DMAR PCI
scope paths. Scope provides enough information to infer context
address, and it is directly matched against DMAR tables scopes.

When calculating bus addresses for the scope or device, use direct
pci_cfgregread(PCIR_SECBUS_1) to get the secondary bus number, instead
of pcib_get_bus().

The issue was observed on HP Gen servers, where iLO PCI devices are
located behind south bridge switch. Turning on translation without
satisfying RMRR requests caused iLO to mostly hang, up to the level of
being unusable to control the server.

While there, remove hw.dmar.dmar_match_verbose tunable, and make the
normal logging under bootverbose useful and sufficient to diagnose
DRHD and RMRR parsing and matching.

Sponsored by: Mellanox Technologies
MFC after: 1 week

show more ...

Convert x86 cache invalidation functions to ifuncs.

This simplifies the runtime logic and reduces the number of
runtime-constant branches.

Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundati

Convert x86 cache invalidation functions to ifuncs.

This simplifies the runtime logic and reduces the number of
runtime-constant branches.

Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
Differential revision: https://reviews.freebsd.org/D16736

show more ...

sys/x86: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error

sys/x86: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

Correct operators precedence.
Also keep the calculated vm_page_alloc_contig() flags in the variable
to not re-evaluate it on the loop iteration.

Noted by: alc
Sponsored by: The FreeBSD Foundation

Replace manyinstances of VM_WAIT with blocking page allocation flags
similar to the kernel memory allocator.

This simplifies NUMA allocation because the domain will be known at wait
time and races b

Replace manyinstances of VM_WAIT with blocking page allocation flags
similar to the kernel memory allocator.

This simplifies NUMA allocation because the domain will be known at wait
time and races between failure and sleeping are eliminated. This also
reduces boilerplate code and simplifies callers.

A wait primitive is supplied for uma zones for similar reasons. This
eliminates some non-specific VM_WAIT calls in favor of more explicit
sleeps that may be satisfied without new pages.

Reviewed by: alc, kib, markj
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon

show more ...

Timeout DMAR commands.

Implement timeouts for register-based DMAR commands. Tunable/sysctl
hw.dmar.timeout specifies the timeout in nanoseconds, set it to zero
to allow infinite wait. Default is 1

Timeout DMAR commands.

Implement timeouts for register-based DMAR commands. Tunable/sysctl
hw.dmar.timeout specifies the timeout in nanoseconds, set it to zero
to allow infinite wait. Default is 1ms.

Runtime modification of the sysctl is not safe, it is allowed for
debugging.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

Add hw.dmar.batch_coalesce tunable/sysctl, which specifies rate at
which queued invalidation completion interrupt is requested with
regard to the queued invalidation requests. In other words, settin

Add hw.dmar.batch_coalesce tunable/sysctl, which specifies rate at
which queued invalidation completion interrupt is requested with
regard to the queued invalidation requests. In other words, setting
the value of the knob to N requests completion interrupt after N items
are processed. Existing behaviour is restored by setting
hw.dmar.batch_coalesce=1.

The knob significantly decreases the DMAR qi interrupt rate at the
cost of slightly longer DMAR map entries recycling.

Sponsored by: The FreeBSD Foundation

show more ...

Split the DMAR unit domains and contexts. Domains carry address space
and related data structures. Contexts attach requests initiators to
domains. There is still 1:1 correspondence between context

Split the DMAR unit domains and contexts. Domains carry address space
and related data structures. Contexts attach requests initiators to
domains. There is still 1:1 correspondence between contexts and
domains on the running system, since only busdma currently allocates
them, using dmar_get_ctx_for_dev().

Large part of the change is formal rename of the ctx to domain, but
patch also reworks the context allocation and free to allow for
independent domain creation.

The helper dmar_move_ctx_to_domain() is introduced for future use, to
reassign request initiator from one domain to another. The hard issue
which is not yet resolved with the context move is proper handling (or
reserving) RMRR entries in the destination domain as required by ACPI
DMAR table for moved context.

Tested by: pho
Sponsored by: The FreeBSD Foundation

show more ...