sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Adjust function definition in aim_machdep.c to avoid clang 15 warning

With clang 15, the following -Werror warning is produced:

sys/powerpc/aim/aim_machdep.c:750:14: error: a function declarati

Adjust function definition in aim_machdep.c to avoid clang 15 warning

With clang 15, the following -Werror warning is produced:

sys/powerpc/aim/aim_machdep.c:750:14: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
mpc745x_sleep()
^
void

This is because mpc745x_sleep() is declared with a (void) argument list,
but defined with an empty argument list. Make the definition match the
declaration.

MFC after: 3 days

show more ...

ppc aim flush_disable_caches: Use void casts instead of a dummy variable.

powerpc64: make radix with superpages default

As Radix MMU with superpages enabled is now stable, make it the
default choice on supported hardware (POWER9 and above), since its
performance is greate

powerpc64: make radix with superpages default

As Radix MMU with superpages enabled is now stable, make it the
default choice on supported hardware (POWER9 and above), since its
performance is greater than that of HPT MMU.

Reviewed by: alfredo, jhibbits
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D30797

show more ...

powerpc64: fix OFWFB with Radix MMU

Current implementation of Radix MMU doesn't support mapping
arbitrary virtual addresses, such as the ones generated by
"direct mapping" I/O addresses. This caused

powerpc64: fix OFWFB with Radix MMU

Current implementation of Radix MMU doesn't support mapping
arbitrary virtual addresses, such as the ones generated by
"direct mapping" I/O addresses. This caused the system to hang, when
early I/O addresses, such as those used by OpenFirmware Frame Buffer,
were remapped after the MMU was up.

To avoid having to modify mmu_radix_kenter_attr just to support this
use case, this change makes early I/O map use virtual addresses from
KVA area instead (similar to what mmu_radix_mapdev_attr does), as
these can be safely remapped later.

Reviewed by: alfredo (earlier version), jhibbits (in irc)
MFC after: 2 weeks
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D31232

show more ...

Create sys/reg.h for the common code previously in machine/reg.h

Move the common kernel function signatures from machine/reg.h to a new
sys/reg.h. This is in preperation for adding PT_GETREGSET to p

Create sys/reg.h for the common code previously in machine/reg.h

Move the common kernel function signatures from machine/reg.h to a new
sys/reg.h. This is in preperation for adding PT_GETREGSET to ptrace(2).

Reviewed by: imp, markj
Sponsored by: DARPA, AFRL (original work)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19830

show more ...

powerpc/aim: Update timebase directly on resume instead of through platform

This only works on single-CPU G4 systems, and more work is needed for
dual-CPU systems. That said, platform sleep does no

powerpc/aim: Update timebase directly on resume instead of through platform

This only works on single-CPU G4 systems, and more work is needed for
dual-CPU systems. That said, platform sleep does not work, and this is
currently only used for PMU-based CPU speed change.

The elimination of the platform_smp_timebase_sync() call is so that the
timebase sync rendezvous can be enhanced to perform better
synchronization, which requires a full rendezvous. This would be
impossible to do on this single-threaded run.

show more ...

powerpc/powermac: Constrain 'cpu_sleep()' for AIM to mpc745x

Rename cpu_sleep() to mpc745x_sleep() to denote what it's actually
intended for. This function is very G4-specific, and will not work on

powerpc/powermac: Constrain 'cpu_sleep()' for AIM to mpc745x

Rename cpu_sleep() to mpc745x_sleep() to denote what it's actually
intended for. This function is very G4-specific, and will not work on
any other CPU. This will afterward eliminate a
platform_smp_timebase_sync() call by directly updating the timebase
instead.

show more ...

powerpc/aim: Add machine check handler for TLB multihit

Handle TLB multi-hit the same as ERAT multi-hit, by flushing the full
TLB.

[PowerPC64LE] LE bringup work: locore / machdep / platform

This is the initial LE changes required in the machdep code to get as far
as platform attachment on qemu pseries.

Sponsored by: Tag1 Consu

[PowerPC64LE] LE bringup work: locore / machdep / platform

This is the initial LE changes required in the machdep code to get as far
as platform attachment on qemu pseries.

Sponsored by: Tag1 Consulting, Inc.

show more ...

[PowerPC64] Hide dssall instruction from llvm assembler

When doing a build for a modern CPUTYPE, llvm will throw errors if obsolete
instructions are used, even if they will never run due to runtime

[PowerPC64] Hide dssall instruction from llvm assembler

When doing a build for a modern CPUTYPE, llvm will throw errors if obsolete
instructions are used, even if they will never run due to runtime checks.

Hiding the dssall instruction from the assembler fixes kernel build when
overriding CPUTYPE, without having any effect on the generated binary.

This has been in my local tree for over a year and is well tested across
a variety of machines.

Sponsored by: Tag1 Consulting, Inc.

show more ...

powerpc: clean up empty lines in .c and .h files

[PowerPC] Fix translation-related crashes during startup

After spending a lot of time trying to track down what was going on, I have
isolated the "black screen" failures when using boot1 to boot a G

[PowerPC] Fix translation-related crashes during startup

After spending a lot of time trying to track down what was going on, I have
isolated the "black screen" failures when using boot1 to boot a G4 machine.

It turns out we were replacing the traps before installing the temporary
BAT entry for the bottom of physical memory. That meant that until the MMU
was bootstrapped, the cached translations were the only thing keeping us
from losing.

Throwing boot1 into the mix was affecting execution flow enough to cause us
to hit an uncached page and crash.

Fix this by properly setting up the initial BAT entry at the same time we
are replacing the OpenFirmware traps, so we can continue executing in
segment 0 until the rest of the DMAP has been set up.

A second thing discovered while researching this is that we were entering a
BAT region for segment 16. It turns out this range was a) considered part
of KVA, and b) has firmware mappings with varying attributes.

If we ever accessed an unmapped page in segment 16, it would cause a BAT
entry to be installed for the whole segment, which would bypass the
existing mappings until it was flushed out again.

Instead, translate the OFW memory attributes into VM memory attributes and
install the ranges into the kernel address space properly.

Reviewed by: adalava
MFC after: 3 weeks
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D25547

show more ...

powerpc: Handle machine checks caused by D-ERAT multihit

Instead of crashing the user process when a D-ERAT multihit is detected, try
to flush the ERAT, and continue. This machine check indicates a

powerpc: Handle machine checks caused by D-ERAT multihit

Instead of crashing the user process when a D-ERAT multihit is detected, try
to flush the ERAT, and continue. This machine check indicates a likely PMAP
invalidation shortcoming that will need to be addressed, but it's
recoverable, so just recover. The recovery is pmap-specific to flush the
ERAT, so add a pmap function to do so, currently only implemented by the
POWER9 radix pmap.

show more ...

powerpc64: Implement Radix MMU for POWER9 CPUs

Summary:
POWER9 supports two MMU formats: traditional hashed page tables, and Radix
page tables, similar to what's presesnt on most other architectures

powerpc64: Implement Radix MMU for POWER9 CPUs

Summary:
POWER9 supports two MMU formats: traditional hashed page tables, and Radix
page tables, similar to what's presesnt on most other architectures. The
PowerISA also specifies a process table -- a table of page table pointers--
which on the POWER9 is only available with the Radix MMU, so we can take
advantage of it with the Radix MMU driver.

Written by Matt Macy.

Differential Revision: https://reviews.freebsd.org/D19516

show more ...

powerpc: Add a CPU-custom machine check handler

Summary:
Some machine checks are process-recoverable, others are not. Let a
CPU-specific handler decide what to do.

This works around a machine chec

powerpc: Add a CPU-custom machine check handler

Summary:
Some machine checks are process-recoverable, others are not. Let a
CPU-specific handler decide what to do.

This works around a machine check error hit while building www/firefox
and mail/thunderbird, which would otherwise cause the build to fail.

More work is needed to handle all possible machine check conditions, but
this is sufficient to unblock some ports building.

Differential Revision: https://reviews.freebsd.org/D23731

show more ...

[PowerPC] kernel ifunc support for powerpc*, fix ppc64 relocation oddities.

This is a general cleanup of the relocatable kernel support on powerpc,
needed to enable kernel ifuncs.

* Fix some reloc

[PowerPC] kernel ifunc support for powerpc*, fix ppc64 relocation oddities.

This is a general cleanup of the relocatable kernel support on powerpc,
needed to enable kernel ifuncs.

* Fix some relocatable issues in the kernel linker, and change to using
a RELOCATABLE_KERNEL #define instead of #ifdef __powerpc__ for parts that
other platforms can use in the future if they wish to have ET_DYN kernels.

* Get rid of the DB_STOFFS hack now that the kernel is relocated to the DMAP
properly across the board on powerpc64.

* Add powerpc64 and powerpc32 ifunc functionality.

* Allow AIM64 virtual mode OF kernels to run from the DMAP like other AIM64
by implementing a virtual mode restart. This fixes the runtime address on
PowerMac G5.

* Fix symbol relocation problems on post-relocation kernels by relocating
the symbol table.

* Add an undocumented method for supplying kernel symbols on powernv and
other powerpc machines using linux-style kernel/initrd loading -- If
you pass the kernel in as the initrd as well, the copy resident in initrd
will be used as a source for symbols when initializing the debugger.
This method is subject to removal once we have a better way of doing this.

Approved by: jhibbits
Relnotes: yes
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D23156

show more ...

[PowerPC] Fix VSX context handling

In r356767, memcpy/memmove/bcopy optimizations were added to libc to
improve performance.

This exposed an existing kernel issue in VSX handling. The PSL_VSX flag

[PowerPC] Fix VSX context handling

In r356767, memcpy/memmove/bcopy optimizations were added to libc to
improve performance.

This exposed an existing kernel issue in VSX handling. The PSL_VSX flag was
not being excluded from the psl_userstatic set, which meant that any thread
that used these and then called swapcontext(3) would get an EINVAL error.

Fixing this exposed a second issue - in r344123, the FPU was being forced
off in set_mcontext(). However, this was neglecting to ensure VSX was turned
off at the same time.

While here, add some code comments to explain what's going on.

Reviewed by: jhibbits, luporl (earlier rev), pkubaj (earlier rev)
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D23497

show more ...

D23057: [PowerPC] Fix offset calculations in bridge mode

In rS354701, I replaced text relocations with offsets from &generictrap.

Unfortunately, the magic variable I was using doesn't actually mean

D23057: [PowerPC] Fix offset calculations in bridge mode

In rS354701, I replaced text relocations with offsets from &generictrap.

Unfortunately, the magic variable I was using doesn't actually mean the
address of &generictrap, in bridge mode it actually means &generictrap64.

So, for bridge mode to work, it is necessary to differentiate between
"where do we need to branch to to handle a trap" and "where is &generictrap
for purposes of doing relative math".

Introduce a new TRAP_ENTRY and use it instead of TRAP_GENTRAP for doing
actual calls to the generic trap handler.

Reported by: Mark Millard <marklmi@yahoo.com>
Reviewed by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D23057

show more ...

powerpc: Add POWER8NVL definition

The POWER8NVL (POWER8 NVLink) architecturally behaves identically to the
POWER8, with a different PVR identifier. Mark it as such, so it shows up
appropriately to

powerpc: Add POWER8NVL definition

The POWER8NVL (POWER8 NVLink) architecturally behaves identically to the
POWER8, with a different PVR identifier. Mark it as such, so it shows up
appropriately to the user.

Reported by: Alexey Kardashevskiy
MFC after: 2 weeks

show more ...

powerpc: Re-merge isa3 HPT with moea64 native HPT

r345402 fixed the bug that led to the split of the ISA 3.0 HPT handling from
the existing manager. The cause of the bug was gcc moving the register

powerpc: Re-merge isa3 HPT with moea64 native HPT

r345402 fixed the bug that led to the split of the ISA 3.0 HPT handling from
the existing manager. The cause of the bug was gcc moving the register
holding VPN to a different register (not r0), which triggered bizarre
behaviors. With the fix, things work, so they can be re-merged. No
performance lost with the merge.

show more ...

ppc64: handle exception 0x1500 (soft patch)

This change adds a hypervisor trap handler for exception 0x1500 (soft patch),
normalizing all VSX registers and returning.
This avoids a kernel panic due

ppc64: handle exception 0x1500 (soft patch)

This change adds a hypervisor trap handler for exception 0x1500 (soft patch),
normalizing all VSX registers and returning.
This avoids a kernel panic due to unknown exception.

Change made with the collaboration of leonardo.bianconi_eldorado.org.br,
that found out that this is a hypervisor exception and not a supervisor one,
and fixed this in the code.

Reviewed by: jhibbits, sbruno
Differential Revision: https://reviews.freebsd.org/D17806

show more ...

Split the PowerISA 3.0 HPT implementation from historic

PowerISA 3.0 makes several changes to not only the format of the HPT but
also the behavior surrounding it. For instance, TLBIE no longer requ

Split the PowerISA 3.0 HPT implementation from historic

PowerISA 3.0 makes several changes to not only the format of the HPT but
also the behavior surrounding it. For instance, TLBIE no longer requires
serialization. Removing this lock cuts buildworld time in half on a
18-core/72-thread POWER9 system, demonstrating that this lock is highly
contended on such a system.

There was odd behavior observed trying to make this change in a
backwards-compatible manner in moea64_native.c, so the best option was to
fully split it, and largely revert the original changes adding POWER9
support to the original file.

Suggested by: nwhitehorn

show more ...

Add hypervisor trap handling, using HSRR0/HSRR1

Summary:
Some hypervisor exceptions on POWER architecture only save state to HSRR0/HSRR1.
Until we have bhyve on POWER, use a lightweight exception fr

Add hypervisor trap handling, using HSRR0/HSRR1

Summary:
Some hypervisor exceptions on POWER architecture only save state to HSRR0/HSRR1.
Until we have bhyve on POWER, use a lightweight exception frontend which copies
HSRR0/HSRR1 into SRR0/SRR1, and run the normal trap handler.

The first user of this is the Hypervisor Virtualization Interrupt, which targets
the XIVE interrupt controller on POWER9.

Reviewed By: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D15487

show more ...

powerpc64: Set n_slbs = 32 for POWER9

Summary:
POWER9 also contains 32 slbs entries as explained by the POWER9 User Manual:

"For HPT translation, the POWER9 core contains a unified (combined for b

powerpc64: Set n_slbs = 32 for POWER9

Summary:
POWER9 also contains 32 slbs entries as explained by the POWER9 User Manual:

"For HPT translation, the POWER9 core contains a unified (combined for both
instruction and data), 32-entry, fully-associative SLB per thread"

Submitted by: Breno Leitao
Differential Revision: https://reviews.freebsd.org/D15128

show more ...