Enable UVM percpu cache on riscv64

Proved stable in multiple ports bulk builds. ok kettenis@ phessler@

remove prototypes for removed functions

clockintr: rename "struct clockintr_queue" to "struct clockqueue"

The code has outgrown the original name for this struct. Both the
external and internal APIs have used the "clockqueue" namespace f

clockintr: rename "struct clockintr_queue" to "struct clockqueue"

The code has outgrown the original name for this struct. Both the
external and internal APIs have used the "clockqueue" namespace for
some time when operating on it, and that name is eyeball-consistent
with "clockintr" and "clockrequest", so "clockqueue" it is.

show more ...

On Allwinner D1, the SBI call to schedule timer interrupts doesn't work.
Instead we have to use one of the timers integerated on the SoC that
triggers an external interrupt. Add the appropriate driv

On Allwinner D1, the SBI call to schedule timer interrupts doesn't work.
Instead we have to use one of the timers integerated on the SoC that
triggers an external interrupt. Add the appropriate driver and change
the MD clock code to hook it up.

ok cheloha@, jca@

show more ...

clockintr: switch from callee- to caller-allocated clockintr structs

Currently, clockintr_establish() calls malloc(9) to allocate a
clockintr struct on behalf of the caller. mpi@ says this behavior

clockintr: switch from callee- to caller-allocated clockintr structs

Currently, clockintr_establish() calls malloc(9) to allocate a
clockintr struct on behalf of the caller. mpi@ says this behavior is
incompatible with dt(4). In particular, calling malloc(9) during the
initialization of a PCB outside of dt_pcb_alloc() is (a) awkward and
(b) may conflict with future changes/optimizations to PCB allocation.

To side-step the problem, this patch changes the clockintr subsystem
to use caller-allocated clockintr structs instead of callee-allocated
structs.

clockintr_establish() is named after softintr_establish(), which uses
malloc(9) internally to create softintr objects. The clockintr subsystem
is no longer using malloc(9), so the "establish" naming is no longer apt.
To avoid confusion, this patch also renames "clockintr_establish" to
"clockintr_bind".

Requested by mpi@. Tweaked by mpi@.

Thread: https://marc.info/?l=openbsd-tech&m=170597126103504&w=2

ok claudio@ mlarkin@ mpi@

show more ...

Import the DVFS code from arm64.

ok jca@, jmatthew@, jsing@

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

show more ...

cpu_idle_{enter,leave} are no-ops on riscv64, so just #define
away the calls

ok jca@

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a comp

statclock: move profil(2), GPROF code to profclock(), gmonclock()

This patch isolates profil(2) and GPROF from statclock(). Currently,
statclock() implements both profil(2) and GPROF through a complex
mechanism involving both platform code (setstatclockrate) and the
scheduler (pscnt, psdiv, and psratio). We have a machine-independent
interface to the clock interrupt hardware now, so we no longer need to
do it this way.

- Move profil(2)-specific code from statclock() to a new clock
interrupt callback, profclock(), in subr_prof.c. Each
schedstate_percpu has its own profclock handle. The profclock is
enabled/disabled for a given CPU when it is needed by the running
thread during mi_switch() and sched_exit().

- Move GPROF-specific code from statclock() to a new clock interrupt
callback, gmonclock(), in subr_prof.c. Where available, each cpu_info
has its own gmonclock handle . The gmonclock is enabled/disabled for
a given CPU via sysctl(2) in prof_state_toggle().

- Both profclock() and gmonclock() have a fixed period, profclock_period,
that is initialized during initclocks().

- Export clockintr_advance(), clockintr_cancel(), clockintr_establish(),
and clockintr_stagger() via <sys/clockintr.h>. They have external
callers now.

- Delete pscnt, psdiv, psratio. From schedstate_percpu, also delete
spc_pscnt and spc_psdiv. The statclock frequency is not dynamic
anymore so these variables are now useless.

- Delete code/state related to the dynamic statclock frequency from
kern_clockintr.c. The statclock frequency can still be pseudo-random,
so move the contents of clockintr_statvar_init() into clockintr_init().

With input from miod@, deraadt@, and claudio@. Early revisions
cleaned up by claudio. Early revisions tested by claudio@. Tested by
cheloha@ on amd64, arm64, macppc, octeon, and sparc64 (sun4v).
Compile- and boot- tested on i386 by mlarkin@. riscv64 compilation
bugs found by mlarkin@. Tested on riscv64 by jca@. Tested on
powerpc64 by gkoehler@.

show more ...

riscv64: switch to clockintr

- Remove riscv64-specific clock interrupt scheduling bits from
cpu_info.
- Remove riscv64-specific randomized statclock() bits from
riscv64/clock.c.
- Remove the 'st

riscv64: switch to clockintr

- Remove riscv64-specific clock interrupt scheduling bits from
cpu_info.
- Remove riscv64-specific randomized statclock() bits from
riscv64/clock.c.
- Remove the 'stat_count' evcount. All clock interrupts are
now counted via the 'clock_count' evcount.
- Wire up timer_intrclock.

With input from jca@, kettenis@. Tested by jca@.

Link: https://marc.info/?l=openbsd-tech&m=166776413003655&w=2

ok kettenis@ jca@ mlarkin@

show more ...

use ansi volatile keyword, not __volatile
ok miod@ guenther@

riscv64: trigger deferred timer interrupts from splx(9)

In order to move riscv64 to a machine-independent clock interrupt
subsystem, the riscv64 clock interrupt code needs to function without
any sp

riscv64: trigger deferred timer interrupts from splx(9)

In order to move riscv64 to a machine-independent clock interrupt
subsystem, the riscv64 clock interrupt code needs to function without
any specific knowledge of the clock interrupt schedule.

The easiest way to achieve this (as we just did with powerpc and
powerpc64) is, if the timer interrupt fires while the CPU is at or
above IPL_CLOCK, defer clock interrupt work until the the timer
interrupt is logically unmasked in splx(9).

In particular, trigger the timer interrupt from plic_setipl() so the
interrupt, if any, is pending before we handle soft interrupts.

Because we're no longer deferring work until the next tick, we don't
need to count pending statclock ticks in struct cpu_info.

kettenis@ notes that the timer triggering code should be refactored
into more generic code when we add support for a non-plic(4) riscv64
machine.

Graciously fixed, compiled, and tested by jca@.

Link: https://marc.info/?l=openbsd-tech&m=165931635410276&w=2

ok kettenis@

show more ...

Implement CPU_BUSY_CYCLE() with the ZiHintPause extension

This may help save power but also improves performance in contended
workloads.

kettenis@ helped a lot:
- pointing out the error in my newes

Implement CPU_BUSY_CYCLE() with the ZiHintPause extension

This may help save power but also improves performance in contended
workloads.

kettenis@ helped a lot:
- pointing out the error in my newest diff
- pointing out that the ZiHintPause had been ratified in early 2021
- testing this on his visionfive board

ok kettenis@

libc make -j4 time before
17m39.06s real 22m13.85s user 34m27.88s system
17m35.60s real 22m15.01s user 34m28.51s system
17m40.78s real 22m14.89s user 34m42.91s system
17m40.61s real 22m06.99s user 34m29.58s system
17m42.79s real 22m08.04s user 34m33.19s system
17m39.21s real 22m11.27s user 34m23.18s system
17m32.81s real 22m13.84s user 34m09.67s system

libc make -j4 time after
16m35.48s real 21m57.94s user 32m05.50s system
16m49.10s real 21m58.47s user 32m42.37s system
16m42.16s real 22m00.51s user 32m22.18s system
16m41.54s real 21m53.32s user 32m19.53s system
16m45.42s real 21m52.14s user 32m37.78s system
16m46.55s real 21m58.98s user 32m36.70s system

show more ...

Remove unused function prototype.

Implement a workaround for the SiFive FU740 CIP-1200 errata.

Remove the (incomplete) support for ASIDs. RISC-V hardware that implements
ASIDs doesn't exist at this moment and the current code inter

Implement a workaround for the SiFive FU740 CIP-1200 errata.

Remove the (incomplete) support for ASIDs. RISC-V hardware that implements
ASIDs doesn't exist at this moment and the current code interferes with
the errata workaround and other pmap improvements we're planning to make.

ok drahn@, jca@, deraadt@

show more ...

Simplify the way we track the FPU state, using powerpc64 as a model.
The new code still uses the clean/dirty state that the hardware reports
to optimize saving/restoring the FPU register, but no long

Simplify the way we track the FPU state, using powerpc64 as a model.
The new code still uses the clean/dirty state that the hardware reports
to optimize saving/restoring the FPU register, but no longer attempts to
keep the FPU registers alive across a context switch. Fixes panics seen
on MP kernels.

ok drahn@

show more ...

SMP support. Mostly works, but occasionally craps out during boot.

ok drahn@

kernel: introduce per-CPU panic(9) message buffers

Add a 512-byte buffer (ci_panicbuf) to each cpu_info struct on each
platform for use by panic(9). The first panic on a given CPU writes
its messag

kernel: introduce per-CPU panic(9) message buffers

Add a 512-byte buffer (ci_panicbuf) to each cpu_info struct on each
platform for use by panic(9). The first panic on a given CPU writes
its message to this buffer. Subsequent panics on a given CPU print
the panic message to the console but do not modify the buffer. This
aids debugging in two cases:

- If 2+ CPUs panic simultaneously there is no risk of garbled messages
in the panic buffer.

- If a CPU panics and then the operator causes a second panic while
using ddb(4), the operator can still recall the first failure on
a particular CPU.

Misc. changes to support this bigger change:

- Set panicstr atomically to identify the first CPU to reach panic().

- Tweak db_show_panic_cmd() to print all panic messages across all
CPUs. Prefix the first panic with an asterisk ('*').

- Prefer db_printf() to printf() during a panic if we have it.
Apparently it disturbs less global state.

- On amd64, tweak fault() to write the local panic buffer. This needs
more work.

Prompted by bluhm@ and deraadt@. Mostly written by deraadt@.
Discussed with bluhm@, deraadt@ and kettenis@.

Borne from a discussion on tech@ about making panic(9) more MP-safe:

https://marc.info/?l=openbsd-tech&m=162086462316143&w=2

ok kettenis@, visa@, bluhm@, deraadt@

show more ...

add OpenBSD rcs ids

whitespace cleanup

fpu_valid_opcode() did not correctly handle 16 bit fp instructions
such as an stval of 0xaa22. The RISC-V Instruction Set Manual states
that setting stval to a non-zero value with the instruction on

fpu_valid_opcode() did not correctly handle 16 bit fp instructions
such as an stval of 0xaa22. The RISC-V Instruction Set Manual states
that setting stval to a non-zero value with the instruction on illegal
instruction exception is an optional feature so instead of changing
fpu_valid_opcode() remove it entirely.

ok deraadt@ kettenis@ drahn@

show more ...

The clock on RISC-V is architectural, so we really don't need the
whole abstraction layer to support multiple timers. And we don't
really need a separate driver. Replace timer(4) with code based on

The clock on RISC-V is architectural, so we really don't need the
whole abstraction layer to support multiple timers. And we don't
really need a separate driver. Replace timer(4) with code based on
the powerpc64 implementation of the randomized statclock code.

Fixes hangs seen on real hardware.

ok jsg@, drahn@

show more ...

reduce diff to current arm64

ok mlarkin@

Initial import of OpenBSD/riscv64

This work is based on the effort:
https://www.openbsd.org/papers/Porting_OpenBSD_to_RISCV_FinalReport.pdf
"Porting OpenBSD to RISC-V ISA"
by
Brian Bamsch <bbamsch@g

Initial import of OpenBSD/riscv64

This work is based on the effort:
https://www.openbsd.org/papers/Porting_OpenBSD_to_RISCV_FinalReport.pdf
"Porting OpenBSD to RISC-V ISA"
by
Brian Bamsch <bbamsch@google.com>
Wenyan He <wenyan.he@sjsu.edu>
Mars Li <mengshi.li.mars@gmail.com>
Shivam Waghela <shivamwaghela@gmail.com>

With additional work by Dale Rahn <drahn@openbsd.org>

show more ...