kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack

Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete

kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack

Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete comment that
was removed from i386 in 2010.

Sponsored by: Netflix
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44069

show more ...

sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

amd64: do not assume that kernel is loaded at 2M physical

Allow any 2M aligned contiguous location below 4G for the staging
area location. It should still be mapped by loader at KERNBASE.

The assu

amd64: do not assume that kernel is loaded at 2M physical

Allow any 2M aligned contiguous location below 4G for the staging
area location. It should still be mapped by loader at KERNBASE.

The assumption kernel makes about loader->kernel handoff with regard to
the MMU programming are explicitly listed at the beginning of hammer_time(),
where kernphys is calculated. Now kernphys is the variable instead of
symbol designating the physical address.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31121

show more ...

Unobfuscate "KERNLOAD" parameter on amd64. This change lines-up amd64 with the
i386 and the rest of supported architectures by defining KERNLOAD in the
vmparam.h and getting rid of magic constant in

Unobfuscate "KERNLOAD" parameter on amd64. This change lines-up amd64 with the
i386 and the rest of supported architectures by defining KERNLOAD in the
vmparam.h and getting rid of magic constant in the linker script, which albeit
documented via comment but isn't programmatically accessible at a compile time.

Use KERNLOAD to eliminate another (matching) magic constant 100 lines down
inside unremarkable TU "copy.c" 3 levels deep in the EFI loader tree.

Reviewed by: markj
Approved by: markj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D27355

show more ...

Tighten mapping protections on preloaded files on amd64.

- We load the kernel at 0x200000. Memory below that address need not
be executable, so do not map it as such.
- Remove references to .ldat

Tighten mapping protections on preloaded files on amd64.

- We load the kernel at 0x200000. Memory below that address need not
be executable, so do not map it as such.
- Remove references to .ldata and related sections in the kernel linker
script. They come from ld.bfd's default linker script, but are not
used, and we now use ld.lld to link the amd64 kernel. lld does not
contain a default linker script.
- Pad the .bss to a 2MB as we do between .text and .data. This
forces the loader to load additional files starting in the following
2MB page, preserving the use of superpage mappings for kernel data.
- Map memory above the kernel image with NX. The kernel linker now
upgrades protections as needed, and other preloaded file types
(e.g., entropy, microcode) need not be mapped with execute permissions
in the first place.

Reviewed by: kib
MFC after: 1 month
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21859

show more ...

Expose the kernel's build-ID through sysctl

After our migration (of certain architectures) to lld the kernel is built
with a unique build-ID. Make it available via a sysctl and uname(1) to
allow th

Expose the kernel's build-ID through sysctl

After our migration (of certain architectures) to lld the kernel is built
with a unique build-ID. Make it available via a sysctl and uname(1) to
allow the user to identify their running kernel.

Submitted by: Ali Mashtizadeh <ali_mashtizadeh.com>
MFC after: 2 weeks
Relnotes: Yes
Event: Waterloo Hackathon 2019
Differential Revision: https://reviews.freebsd.org/D20326

show more ...

Add comment to explain kernel ldscript 0x200000 constant

Reported by: linimon

amd64: tweak the read_frequently section

1. align to 128 bytes to avoid possible waste from the preceeding section
2. sort entries by alignment SORT_BY_ALIGNMENT, plugging the holes (most
entries ar

amd64: tweak the read_frequently section

1. align to 128 bytes to avoid possible waste from the preceeding section
2. sort entries by alignment SORT_BY_ALIGNMENT, plugging the holes (most
entries are one byte in size, but they got interleaved with bigger ones)

Interestingly I was looking for a feature of the sort earlier and failed
to find it. It turns out the script was already utilizing sorting in other
places, so shame on me.

Thanks for Travis Geiselbrecht for pointing me at the feature.

show more ...

amd64: align the .data.exclusive_cache_line section to 128

This aligns the section itself compared to other sections, does not change
internal alignment of fields stored inside. This may or may not

amd64: align the .data.exclusive_cache_line section to 128

This aligns the section itself compared to other sections, does not change
internal alignment of fields stored inside. This may or may not come later.

The motivation is partially combating adverse effects of the adjacent cache
line prefetcher. Without the annotation part of read_mostly section was on
the line of fire.

show more ...

amd64: Protect the kernel text, data, and BSS by setting the RW/NX bits
correctly for the data contained on each memory page.

There are several components to this change:
* Add a variable to indica

amd64: Protect the kernel text, data, and BSS by setting the RW/NX bits
correctly for the data contained on each memory page.

There are several components to this change:
* Add a variable to indicate the start of the R/W portion of the
initial memory.
* Stop detecting NX bit support for each AP. Instead, use the value
from the BSP and, if supported, activate the feature on the other
APs just before loading the correct page table. (Functionally, we
already assume that the BSP and all APs had the same support or
lack of support for the NX bit.)
* Set the RW and NX bits correctly for the kernel text, data, and
BSS (subject to some caveats below).
* Ensure DDB can write to memory when necessary (such as to set a
breakpoint).
* Ensure GDB can write to memory when necessary (such as to set a
breakpoint). For this purpose, add new MD functions gdb_begin_write()
and gdb_end_write() which the GDB support code can call before and
after writing to memory.

This change is not comprehensive:
* It doesn't do anything to protect modules.
* It doesn't do anything for kernel memory allocated after the kernel
starts running.
* In order to avoid excessive memory inefficiency, it may let multiple
types of data share a 2M page, and assigns the most permissions
needed for data on that page.

Reviewed by: jhb, kib
Discussed with: emaste
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D14282

show more ...

Introduce __read_frequently

While __read_mostly groups variables together, their placement is not
specified. In particular 2 frequently used variables can end up in
different lines.

This annotation

Introduce __read_frequently

While __read_mostly groups variables together, their placement is not
specified. In particular 2 frequently used variables can end up in
different lines.

This annotation is only expected to be used for variables read all the time,
e.g. on each syscall entry.

MFC after: 1 week

show more ...

use INT3 instead of NOP for x86 binary padding

We should never end up executing the inter-function padding, so we
are better off faulting than silently carrying on to whatever function
happens to be

use INT3 instead of NOP for x86 binary padding

We should never end up executing the inter-function padding, so we
are better off faulting than silently carrying on to whatever function
happens to be next.

Note that LLD will soon do this by default (although it currently pads
with zeros).

Reviewed by: dim, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10047

show more ...

Introduce __read_mostly and __exclusive_cache_line macros.

The intended use is to annotate frequently used globals which either rarely
change (and thus can be grouped in the same cacheline) or are a

Introduce __read_mostly and __exclusive_cache_line macros.

The intended use is to annotate frequently used globals which either rarely
change (and thus can be grouped in the same cacheline) or are an atomic counter
(which means it may benefit from being the only variable in the cacheline).

Linker script support is provided only for amd64. Architectures without it risk
having other variables put in, i.e. as if they were not annotated. This is
harmless from correctness point of view.

Reviewed by: bde (previous version)
MFC after: 1 month

show more ...

Use explicit 0x200000 instead of MAXPAGESIZE for the amd64 kernel physaddr

MAXPAGESIZE is not well defined by the GNU ld documentation.
Different linkers, and different versions of the same linker,

Use explicit 0x200000 instead of MAXPAGESIZE for the amd64 kernel physaddr

MAXPAGESIZE is not well defined by the GNU ld documentation.
Different linkers, and different versions of the same linker, use
different MAXPAGESIZE values. Current versions of GNU gold and LLVM's
lld use 4K. When set to 4K the kernel panics at boot due to an issue
with x86bios.

Here we want the kernel physaddr to be the amd64 superpage size, so use
that value (2MB) explicitly. With this change GNU gold and LLVM lld can
link a working amd64 kernel.

PR: 214718 (x86bios)
Differential Revision: https://reviews.freebsd.org/D8610

show more ...

remove CONSTRUCTORS from kernel linker scripts

The linker script CONSTRUCTORS keyword is only meaningful "when linking
object file formats which do not support arbitrary sections, such as
ECOFF and

remove CONSTRUCTORS from kernel linker scripts

The linker script CONSTRUCTORS keyword is only meaningful "when linking
object file formats which do not support arbitrary sections, such as
ECOFF and XCOFF"[1] and is ignored for other object file formats.

LLVM's lld does not yet accept (and ignore) CONSTRUCTORS, so just remove
CONSTRUCTORS from the linker scripts as it has no effect.

[1] https://sourceware.org/binutils/docs/ld/Output-Section-Keywords.html

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7343

show more ...

amd64: set the correct LMA values

The current linker script generates program headers with VMA == LMA:

Entry point 0xffffffff802e7000
There are 6 program headers, starting at offset 64

Program Hea

amd64: set the correct LMA values

The current linker script generates program headers with VMA == LMA:

Entry point 0xffffffff802e7000
There are 6 program headers, starting at offset 64

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0xffffffff80200040 0xffffffff80200040
0x0000000000000150 0x0000000000000150 R E 8
INTERP 0x0000000000000190 0xffffffff80200190 0xffffffff80200190
0x000000000000000d 0x000000000000000d R 1
[Requesting program interpreter: /red/herring]
LOAD 0x0000000000000000 0xffffffff80200000 0xffffffff80200000
0x00000000010559b0 0x00000000010559b0 R E 200000
LOAD 0x0000000001056000 0xffffffff81456000 0xffffffff81456000
0x0000000000132638 0x000000000052ecf8 RW 200000
DYNAMIC 0x0000000001056000 0xffffffff81456000 0xffffffff81456000
0x00000000000000d0 0x00000000000000d0 RW 8
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RWE 8

This is fine for the FreeBSD loader, because it completely ignores p_paddr
and instead uses p_vaddr with a hardcoded offset. Other loaders however
acknowledge p_paddr (like the Xen ELF loader), in which case they will try
to load the kernel at the wrong place. Fix this by adding an AT keyword to
the first section specifying the physical address, other sections will
follow suit, so it ends up looking like:

Entry point 0xffffffff802e7000
There are 6 program headers, starting at offset 64

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0xffffffff80200040 0x0000000000200040
0x0000000000000150 0x0000000000000150 R E 8
INTERP 0x0000000000000190 0xffffffff80200190 0x0000000000200190
0x000000000000000d 0x000000000000000d R 1
[Requesting program interpreter: /red/herring]
LOAD 0x0000000000000000 0xffffffff80200000 0x0000000000200000
0x00000000010559b0 0x00000000010559b0 R E 200000
LOAD 0x0000000001056000 0xffffffff81456000 0x0000000001456000
0x0000000000132638 0x000000000052ecf8 RW 200000
DYNAMIC 0x0000000001056000 0xffffffff81456000 0x0000000001456000
0x00000000000000d0 0x00000000000000d0 RW 8
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RWE 8

Tested on bare metal using the native FreeBSD loader and grub2 from TRUEOS.

Sponsored by: Citrix Systems R&D
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D2783

show more ...

The new binutils has correctly redefined MAXPAGESIZE on amd64 as 0x200000
instead of 0x100000. As a side effect, an amd64 kernel now loads at
physical address 0x200000 instead of 0x100000. This is

The new binutils has correctly redefined MAXPAGESIZE on amd64 as 0x200000
instead of 0x100000. As a side effect, an amd64 kernel now loads at
physical address 0x200000 instead of 0x100000. This is probably for the
best because it avoids the use of a 2MB page mapping for the first 1MB of
the kernel that also spans the fixed MTRRs. However, getmemsize() still
thinks that the kernel loads at 0x100000, and so the physical memory between
0x100000 and 0x200000 is lost. Fix this problem by replacing the hard-wired
constant in getmemsize() by a symbol "kernphys" that is defined by the
linker script.

In collaboration with: kib

show more ...

Sync sys/conf/ldscript.amd64 with the upstream version, preserving
FreeBSD-specific customizations (in particular, the addition of
_start_ctors and _stop_ctors).

Binutils commit 0c845abb5a0083c6deebc75975608237015badba increased
ELF_MAXPAGESIZE for amd64 from 0x00100000 to 0x00200000. This caused
the kernel to be incorrectly linked, using the existing linker

Binutils commit 0c845abb5a0083c6deebc75975608237015badba increased
ELF_MAXPAGESIZE for amd64 from 0x00100000 to 0x00200000. This caused
the kernel to be incorrectly linked, using the existing linker script,
resulting in a virtual address of 0xffffffff80000000 for the LOAD
program header.

The boot loader will load such a kernel at a real address of 0x00000000,
which either causes protection faults in btx, crashes the machine, or
(in case of a VMware guest) even makes it power down. :)

Fix this by partially synchronizing the amd64 linker script with
binutils own updated version, in particular replacing a hardcoded
value of 0x00100000 by CONSTANT(MAXPAGESIZE).

show more ...

Use new output format 'elf64-x86-64-freebsd' instead of 'elf64-x86-64',
and similarly 'elf64-sparc-freebsd' instead of 'elf64-sparc'.

Provide the _start_ctors and _stop_ctors symbols. As on i386, the addresses
of these are the start and end of the .ctors section.

Sync up with the files in the hammer branch in the p4 tree to get basic
AMD64 support. There is still more to add.

Add two symbols start_ctors and stop_ctors to allow us to find the
.ctors section so we can call the constructors.

Use the new freebsd output format from Binutils 2.13.1.

Remove hard coded magic load address. Now to change the load address,
we just have to change the pmap.h constants and ld will automatically
adapt based on the "kernbase" symbol.