Avoid using 1GB mappings for the identity map that we use in the early
kernel bootstrap phase and when booting the secondary CPUs. This avoids
accidentally mapping memory regions that should not be

Avoid using 1GB mappings for the identity map that we use in the early
kernel bootstrap phase and when booting the secondary CPUs. This avoids
accidentally mapping memory regions that should not be mapped (i.e. secure
memory) as all mapped memory can be accessed speculatively.

To allow the kernel to be loaded at arbitrary physical addresses (and
arbitrary virtual addresses in the future) make sure the code that builds
the page tables can handle mapping a block of memory that straddles a
1GB or 512GB boundary. This involves using two pagetable pages at all
levels except the top level and changing the link_l1_pagetable function
to accept an entry count just like link_l0_pagetable already does.

ok patrick@

show more ...

Simplify early kernel bootstrap a bit more. Just map the entire 64MB memory
block that the bootloader allocates for us (minus the first 2MB). This means
we can get rid of a bunch of code in pmap_bo

Simplify early kernel bootstrap a bit more. Just map the entire 64MB memory
block that the bootloader allocates for us (minus the first 2MB). This means
we can get rid of a bunch of code in pmap_bootstrap(). It also makes sure
that we don't accidentally enter mappings that cover secure memory just
beyond where the kernel was loaded.

ok patrick@

show more ...

_C_LABEL() and _ASM_LABEL() are no longer useful in the "everything
is ELF" world. Eliminate use of them in amd64, arm64, armv7, i386,
macppc, mips64, and sparc64 code.

ok deraadt@ jca@ krw@

spelling

Introduce BUS_SPACE_MAP_POSTED such that we can distinguish between
posted and non-posted device memory mappings and set the right memory
attributes for them. Needed because on the Apple M1 using th

Introduce BUS_SPACE_MAP_POSTED such that we can distinguish between
posted and non-posted device memory mappings and set the right memory
attributes for them. Needed because on the Apple M1 using the wrong
mapping will fault.

ok patrick@, dlg@

show more ...

On bigger arm64 machines with plenty of memory it is possible that
the kernel, which is allocated in low memory by efiboot, is not in
the same 512 GiB L0 address space as the EFI/FDT/ACPI tables. Si

On bigger arm64 machines with plenty of memory it is possible that
the kernel, which is allocated in low memory by efiboot, is not in
the same 512 GiB L0 address space as the EFI/FDT/ACPI tables. Since
we only identity-map a single L0 entry for bootstrapping purposes we
can't reach those tables. Keep a set of spare L1 tables, currently
one, which we can dynamically enter into the identity map so that we
are able to reach those tables. While there, do the FDT mapping a
bit later in C code by calling pmap_map_early(). This allows us to
boot further on Packet's Ampere eMAGs.

Discussed with drahn@
ok kettenis@

show more ...

Move some code back from locore0.S to locore.S that we will need to bring up
secondary CPUs. This involves adjusting get_virt_delta() to account for the
fact that it may no longer live near the star

Move some code back from locore0.S to locore.S that we will need to bring up
secondary CPUs. This involves adjusting get_virt_delta() to account for the
fact that it may no longer live near the start of kernel code. Heavily based
on a diff from drahn@.

ok patrick@, deraadt@

show more ...

Tighten the permissions used in the early stage page tables somewhat.
Add an interface to establish additional VA=PA 1G block mappings for use by
upcoming EFI runtime services support.

ok guenther@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the ke

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

show more ...