arm64: Check DMAP address is valid in PHYS_IN_DMAP

When checking if a physical address is in the DMAP region we assume
all physical addresses between DMAP_MIN_PHYSADDR and DMAP_MAX_PHYSADDR
are able

arm64: Check DMAP address is valid in PHYS_IN_DMAP

When checking if a physical address is in the DMAP region we assume
all physical addresses between DMAP_MIN_PHYSADDR and DMAP_MAX_PHYSADDR
are able to be accesses through the DMAP. It may be the case that
there is device memory in this range that shouldn't be accessed through
the DMAP mappings.

Add a check to PHYS_IN_DMAP that the translated virtual address is a
valid kernel address. To support code that already checks the address
is valid add PHYS_IN_DMAP_RANGE.

PR: 278233
Reviewed by: alc, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D44677

show more ...

arm64: Move curthread setup earlier

In 469cfa3c30ee cperciva added TSLOG profiling to link_elf_ireloc. This
requires curthread to be read when the kernel linker is invoked, but it
hadn't yet been i

arm64: Move curthread setup earlier

In 469cfa3c30ee cperciva added TSLOG profiling to link_elf_ireloc. This
requires curthread to be read when the kernel linker is invoked, but it
hadn't yet been initialized. On amd64 this was harmless since [gs:0] was
readable; but on arm64 this broke since [x18] was not readable.

Move the curthread (and associated PCPU) setup earlier on arm64 in order
to allow TSLOG to work there.

Fixes: 469cfa3c30ee ("tslog: Annotate some early boot functions")
Differential Revision: https://reviews.freebsd.org/D44317

show more ...

arm64: Use void pointers for arguments to arm64_get_writable_addr

No functional change, but this reduces diffs with CheriBSD downstream.

Reviewed by: andrew
Sponsored by: University of Cambridge, G

arm64: Use void pointers for arguments to arm64_get_writable_addr

No functional change, but this reduces diffs with CheriBSD downstream.

Reviewed by: andrew
Sponsored by: University of Cambridge, Google, Inc.
Differential Revision: https://reviews.freebsd.org/D44344

show more ...

arm64: Fix the has_hyp check

boot_el is now the CurrentEL.EL field value. Use the new macros to
implement the has_hyp() check.

Sponsored by: Arm Ltd

arm64: Add in_vhe() to find if the kernel is in VHE

Add a function to support devices that may need to know if the kernel
has enabled the Armv8.1 Virtulization Host Extensions (FEAT_VHE).
Some devic

arm64: Add in_vhe() to find if the kernel is in VHE

Add a function to support devices that may need to know if the kernel
has enabled the Armv8.1 Virtulization Host Extensions (FEAT_VHE).
Some devices, e.g. the generic timer, will need to know, e.g. use a
different interrupt.

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43973

show more ...

arm: Clean up socdev_va

Support socdev_va on arm and ensure the variable is available on arm64.

Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43359

arm64: Add pmap integration for KMSAN

- In pmap_bootstrap_san(), allocate the root PTPs for the shadow maps.
(For KASAN, this is done earlier since we need to do some special
bootstrapping for t

arm64: Add pmap integration for KMSAN

- In pmap_bootstrap_san(), allocate the root PTPs for the shadow maps.
(For KASAN, this is done earlier since we need to do some special
bootstrapping for the kernel stack.)
- Adjust ifdefs to include KMSAN.
- Expand the shadow maps when pmap_growkernel() is called.

MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D43405

show more ...

arm64: Enable KMSAN when configured to do so

MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remov

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

arm64: Make kern_delta unneeded in the boot params

Use pmap_early_vtophys to translate from a virtual to physical where
we were previously using the calculated delta. This means that, while
we still

arm64: Make kern_delta unneeded in the boot params

Use pmap_early_vtophys to translate from a virtual to physical where
we were previously using the calculated delta. This means that, while
we still calculate it, we don't need to pass it to initarm or either
pmap bootstrap functions.

While here remove an unneeded printf that indirectly used it or was
related to the previous printf.

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D42567

show more ...

arm64: Use pmap_early_vtophys in pmap_bootstrap_san

Use pmap_early_vtophys to find the physical address of the kernel base
rather than using the calculated offset as it will be removed in a
latter c

arm64: Use pmap_early_vtophys in pmap_bootstrap_san

Use pmap_early_vtophys to find the physical address of the kernel base
rather than using the calculated offset as it will be removed in a
latter commit.

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D42566

show more ...

arm64, risvc: warn about ignored kstack_pages for thread0

Suggested by: brooks
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D42143

arm64, riscv: Use KSTACK_PAGES for the thread0 kstack size designator

instead of kstack_pages. Although it is correct right now to use
kstack_pages on amd64 since the kern.kstack_pages tunable is no

arm64, riscv: Use KSTACK_PAGES for the thread0 kstack size designator

instead of kstack_pages. Although it is correct right now to use
kstack_pages on amd64 since the kern.kstack_pages tunable is not
functional on arm64, this is too fragile and wrong on riscv.

Include opt_kstack_pages.h into machdep.c to get the right definition
for KSTACK_PAGES.

Reviewed by: jhb, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D42143

show more ...

arm64 makectx: Fix overflow of tf_x array

PCB_LR isn't stored in tf_x, so trying to store it as pcb_x[PCB_LR] =
tf->tf_x[PCB_LR + PCB_X_START] overflowed the tf_x array.

Reported by: Morello (bound

arm64 makectx: Fix overflow of tf_x array

PCB_LR isn't stored in tf_x, so trying to store it as pcb_x[PCB_LR] =
tf->tf_x[PCB_LR + PCB_X_START] overflowed the tf_x array.

Reported by: Morello (bounds check crash)
Reviewed by: jrtc27, andrew, markj
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D41485

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Add padding for future use on arm64

Allow new features to be supported without changing the size of
existing structures.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://review

Add padding for future use on arm64

Allow new features to be supported without changing the size of
existing structures.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D39777

show more ...

Use a uint64_t to store the arm64 mpidr

Use a single uint64_t to hole the mpidr register as we can break the
KBI on 14. Keep the macro so code can still be MFCd to 13.

Sponsored by: Arm Ltd

Make arm64 pcb padding explicit

There is padding between some fields. Mark those I have found so they
can be reused later if needed.

Sponsored by: Arm Ltd

arm64: Ensure that thread0's PCB flags are initialized

On arm64, the PCB is stored at the top of the thread stack. For thread0
this comes from the static "initstack" region, which is placed in the

arm64: Ensure that thread0's PCB flags are initialized

On arm64, the PCB is stored at the top of the thread stack. For thread0
this comes from the static "initstack" region, which is placed in the
.init_pagetable section, which is not part of the BSS and thus doesn't
get zeroed by locore. (See the comment in ldscript.arm64.) It is thus
possible for the pcb_flags field to be uninitialized, which can result
in PCB_SINGLE_STEP being set.

Fix this by simply initializing the field. A separate commit will move
initstack out of the .init_pagetable section, since it has no reason to
be there, but it is preferable to explicitly initialize PCB fields
anyway. In particular, regular kernel stacks are not zeroed upon
allocation, so we should be consistent here.

Reviewed by: andrew
MFC after: 1 week
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D39343

show more ...

arm64: add KASAN support

This entails:
- Marking some obvious candidates for __nosanitizeaddress
- Similar trap frame markings as amd64, for similar reasons
- Shadow map implementation

The shadow m

arm64: add KASAN support

This entails:
- Marking some obvious candidates for __nosanitizeaddress
- Similar trap frame markings as amd64, for similar reasons
- Shadow map implementation

The shadow map implementation is roughly similar to what was done on
amd64, with some exceptions. Attempting to use available space at
preinit_map_va + PMAP_PREINIT_MAPPING_SIZE (up to the end of that range,
as depicted in the physmap) results in odd failures, so we instead
search the physmap for free regions that we can carve out, fragmenting
the shadow map as necessary to try and fit as much as we need for the
initial kernel map. pmap_bootstrap_san() is thus after
pmap_bootstrap(), which still included some technically reserved areas
of the memory map that needed to be included in the DMAP.

The odd failure noted above may be a bug, but I haven't investigated it
all that much.

Initial work by mhorne with additional fixes from kevans and markj.

Reviewed by: andrew, markj
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D36701

show more ...

Remove unused registes from the arm pcb

These were kept for ABI reasons. Remove them and bump __FreeBSD_version
so debuggers can be updated to use the new layout.

Reviewed by: jhb
Sponsored by: Arm

Remove unused registes from the arm pcb

These were kept for ABI reasons. Remove them and bump __FreeBSD_version
so debuggers can be updated to use the new layout.

Reviewed by: jhb
Sponsored by: Arm Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35378

show more ...

arm64: limit EFI excluded regions to physical memory types

Consolidate add_efi_map_entry() and exclude_efi_map_entry() into a
single function, handle_efi_map_entry(), so that the exact set of entry

arm64: limit EFI excluded regions to physical memory types

Consolidate add_efi_map_entry() and exclude_efi_map_entry() into a
single function, handle_efi_map_entry(), so that the exact set of entry
types handled is the same in the addition or exclusion cases. Before,
exclude_efi_map_entry() had a 'default' case that would exclude all
entry types that were not listed explicitly in the switch statement.

Logically, we do not need to exclude a range that could not possibly be
added to physmem, and we do not need to exclude bus ranges that are not
physical memory, for example EFI_MD_TYPE_IOMEM.

Since physmem's ram0 device will reserve bus memory resources for its
owned ranges, this was preventing attachment of the watchdog device on
the RPI4B. For some reason its region of memory-mapped I/O appeared in
the EFI memory map (with the aforementioned EFI_MD_TYPE_IOMEM type).
This change fixes the attachment issue, as we prevent the physmem API
from messing with this range of bus space.

PR: 270044
Reported by: karels, Mark Millard
Reviewed by: andrew, karels, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39003

show more ...

arm64: disable the physical timer for now if HCR_EL2.E2H is set

On some hardware, we can't clear HCR_EL2.E2H so accesses to the physical
timer hopelessly trap to EL2. Stash off the value of HCR_EL2

arm64: disable the physical timer for now if HCR_EL2.E2H is set

On some hardware, we can't clear HCR_EL2.E2H so accesses to the physical
timer hopelessly trap to EL2. Stash off the value of HCR_EL2 and use it
in has_hyp() to avoid this.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D38884

show more ...

abstract out the vm detection via smbios..

This makes the detection of VMs common between platforms that
have SMBios.

Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D38800

arm64: Fix thread0.td_kstack_pages init

Commit 86a994d6537d7b5e1efb1019e466d86a688fd570 initialized
thread0.td_kstack_pages to KSTACK_PAGES. Due to the lack of an
include of opt_kstack_pages.h it u

arm64: Fix thread0.td_kstack_pages init

Commit 86a994d6537d7b5e1efb1019e466d86a688fd570 initialized
thread0.td_kstack_pages to KSTACK_PAGES. Due to the lack of an
include of opt_kstack_pages.h it used the fallback value of 4 from
machine/param.h. This meant that increasing KSTACK_PAGES in the kernel
config resulted in a panic in _epoch_enter_preempt as the following
assertion was false during network stack setup:

MPASS((vm_offset_t)et >= td->td_kstack &&
(vm_offset_t)et + sizeof(struct epoch_tracker) <=
td->td_kstack + td->td_kstack_pages * PAGE_SIZE);

Switch to initializing with kstack_pages following other architectures.

Reviewed by: imp, markj
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D38048

show more ...