kernel - Fix atomicy issue in APIC EOI and enable_icus

* Change ICU_INTREN() and ICU_INTRDIS() to only modify the ICU
whos bit is being changed, rather than always setting the mask
for both ICUs

kernel - Fix atomicy issue in APIC EOI and enable_icus

* Change ICU_INTREN() and ICU_INTRDIS() to only modify the ICU
whos bit is being changed, rather than always setting the mask
for both ICUs.

* If masking a level IRQ on the APIC, make sure the EOI
to the APIC is atomic with the masking operation.

* Make sure that ICU enablement is atomic with ICU masking
operations.

* Does not fix any known bugs

show more ...

kernel - Intel user/kernel separation MMU bug fix part 1/3

* Part 1/3 of the fix for the Intel user/kernel separation MMU bug.
It appears that it is possible to discern the contents of kernel
me

kernel - Intel user/kernel separation MMU bug fix part 1/3

* Part 1/3 of the fix for the Intel user/kernel separation MMU bug.
It appears that it is possible to discern the contents of kernel
memory with careful timing measurements of instructions due to
speculative memory reads and speculative instruction execution
by Intel cpus. This can happen because Intel will allow both to
occur even when the memory access is later disallowed due to
privilege separation in the PTE.

Even though the execution is always aborted, the speculative
reads and speculative execution results in timing artifacts which
can be measured. A speculative compare/branch can lead to timing
artifacts that allow the actual contents of kernel memory to be
discerned.

While there are multiple speculative attacks possible, the Intel
bug is particularly bad because it allows a user program to more
or less effortlessly access kernel memory (and if a DMAP is
present, all of physical memory).

* Part 1 implements all the logic required to load an 'isolated'
version of the user process's PML4e into %cr3 on all user
transitions, and to load the 'normal' U+K version into %cr3 on
all transitions from user to kernel.

* Part 1 fully allocates, copies, and implements the %cr3 loads for
the 'isolated' version of the user process PML4e.

* Part 1 does not yet actually adjust the contents of this isolated
version to replace the kernel map with just a trampoline map in
kernel space. It does remove the DMAP as a test, though. The
full separation will be done in part 3.

show more ...

kernel: Include generic headers which will take care of platforms.

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

AMD64 - Fix format conversions and other warnings.

AMD64 - Sync machine-dependent bits from smtms.

Submitted-by: Jordan Gordeev <jgordeev@dir.bg>

Use same interrupt vector handler for fast/slow interrupt handlers

Slow interrupt vector handler is removed. Fast interrupt vector handler,
ithread_fast_handler(), now schedules slow interrupt hand

Use same interrupt vector handler for fast/slow interrupt handlers

Slow interrupt vector handler is removed. Fast interrupt vector handler,
ithread_fast_handler(), now schedules slow interrupt handlers if necessary:
o No fast interrupt handlers are registered
o Mixed fast and slow interrpt handlers are registered
o Non-MPSAFE fast interrupt handlers could not get BGL

i386/amd64: gd_ipending field in mdglobaldata is revoked, which is only
used by slow interrupt vector handler.

ithread_fast_handler()'s invoking convetion is changed:
- ithead_fast_handler() must be called with critical section being held
- Callers of ithead_fast_handler() no longer bump gd_intr_nesting_level

Discussed-with: dillon@
Reviewed-by: dillon@

show more ...

AMD64 - Make signals operational, fix reg mappings, fix %fs management, trace

Adjust sigframe, trapframe, mcontext, ucontext, and regs. Add tf_xflags
too all structures. Reorder struct regs to mat

AMD64 - Make signals operational, fix reg mappings, fix %fs management, trace

Adjust sigframe, trapframe, mcontext, ucontext, and regs. Add tf_xflags
too all structures. Reorder struct regs to match the register layout
in the other structures.

Implement the commented out signaling code. Signals now work, or at least
do not crash programs. Theoretically the FP state is also saved and restored.

The exec() code failed to adjust gd_user_fs and gd_user_gs when setting
the msr registers for the user %fs and %gs, causing %fs to unexpectedly
change in running user programs.

Implement trace/debug support functions to set %rip and to single-step.

Define the missing vkernel flag FP_SOFTFP.

show more ...

AMD64 - Sync AMD64 support from Jordan Gordeev's svn repository and
Google SOC project. This work is still continuing but represents
substantial progress in the effort.

With this commit the world b

AMD64 - Sync AMD64 support from Jordan Gordeev's svn repository and
Google SOC project. This work is still continuing but represents
substantial progress in the effort.

With this commit the world builds and installs, the loader is able to
boot the kernel, and the kernel is able to initialize, probe devices, and
exec the init program. The init program is then able to run until it hits
its first fork(). For the purposes of the GSOC the project is being
considered a big success!

The code has been adapted from multiple sources, most notably Peter Wemm
and other peoples work from FreeBSD, with many modifications to make it
work with DragonFly. Also thanks go to Simon Schubert for working on gdb
and compiler issues, and to Noah Yan for a good chunk of precursor work
in 2007.

While Jordan wishes to be modest on his contribution, frankly we would
not have been able to make this much progress without the large number
of man-hours Jordan has dedicated to his GSOC project painstakingly gluing
code together, tracking down issues, and progressing the boot sequence.

Submitted-by: Jordan Gordeev <jgordeev@dir.bg>

show more ...

x86_64: 64-bit index register should be used.

Looks like qemu does not accept 32-bit index register, while the
real boxs and virtualbox accept 32-bit index regiter.

However, according to AMD <<2459

x86_64: 64-bit index register should be used.

Looks like qemu does not accept 32-bit index register, while the
real boxs and virtualbox accept 32-bit index regiter.

However, according to AMD <<24593--Rev. 3.17--June 2010>> Page 25,
64-bit index register should be used to create effective address.

DragonFly-bug: http://bugs.dragonflybsd.org/issue1991

show more ...

icu: Put ICU_IMR_OFFSET into machine_base/icu/icu.h

x86_64 intr: Support upto 192 IDT entries in ipl and intr vector asm code

Most parts are same as following commit on i386:
c263294b570bc9641fe5184b066fd801803046a4
except that 64bits mask array is u

x86_64 intr: Support upto 192 IDT entries in ipl and intr vector asm code

Most parts are same as following commit on i386:
c263294b570bc9641fe5184b066fd801803046a4
except that 64bits mask array is used.

Things like (1UL << $const_val) does not work in .s file; currently
"movq $1,%rcx; shlq $const_val,%rcx;" is used instead.

show more ...

icu: Remove unused macros

x86_64 intr: Don't pass the vector name to INTR_HANDLER

x86_64 intr: We no longer have the fast version of intr vectors

x86_64 - Get completely rid of APIC_IO

For SMP kernels compile time APIC_IO option has been superseeded
by loader tunable hw.apic_io_enable which defaults to APIC I/O
enabled.

kernel - Fix serious issue w/ smp_invltlb(), plus other issues (2)

It looks like there are a number of additional cases where kernel threads
can wind up running with interrupts physically disabled,

kernel - Fix serious issue w/ smp_invltlb(), plus other issues (2)

It looks like there are a number of additional cases where kernel threads
can wind up running with interrupts physically disabled, which can create
serious problems for smp_invltlb().

It is not entirely clear how these deadlocks occur since the IPI code does
a forced "STI" if it would otherwise loop, but there are several other
hard loops that do not: lwkt_switch looping on a vm_token, spin locks,
and probably a few other places.

We want interrupts to be enabled in all cases where these sorts of loops
occur in order to be able to service Xinvltlb via smp_invltlb() as well
as to prevent the LAPIC interrupt queue from filling up and livelocking
something that it shouldn't.

* lwkt_preempt() must save, zero, and restore gd->gd_intr_nesting_level
when issuing a preemption. Otherwise it will improperly panic on
an assertion if the preempting interrupt thread code tries to switch out.
It is perfectly acceptable for the preempting thread to block (it just
switches back to the thread that got preempted).

Why the assertion was not occuring before I do not know but it is
probably related to threads getting stuck in 'cli' mode. The additional
changes below appear to significantly increase the number of interrupt
thread preemptions which succeed (lwkt.preempt_{hit,miss} counters).

* STI prior to running ithread_fast_handler() from all IDTVECs related
to device interrupts.

* STI in Xcpustop, Xipiq, and Xtimer. These functions can call more
complex C code and doing so with interrupts disabled may prevent
Xinvltlb (via smp_invltlb()) from being executed, deadlocking the
system.

* Reorder a mfence(). Probably not needed but do it anyway.

show more ...

kernel - (mainly x86_64) - Fix a number of rare races

* Move the MP lock from outside to inside exit1(), also fixing an issue
where sigexit() was calling exit1() without it.

* Move calls to dsche

kernel - (mainly x86_64) - Fix a number of rare races

* Move the MP lock from outside to inside exit1(), also fixing an issue
where sigexit() was calling exit1() without it.

* Move calls to dsched_exit_thread() and biosched_done() out of the
platform code and into the mainline code. This also fixes an
issue where the code was improperly blocking way too late in the
thread termination code, after the point where it had been descheduled
permanently and tsleep decomissioned for the thread.

* Cleanup and document related code areas.

* Fix a missing proc_token release in the SIGKILL exit path.

* Fix FAKE_MCOUNT()s in the x86-64 code. These are NOPs anyway
(since kernel profiling doesn't work), but fix them anyway.

* Use APIC_PUSH_FRAME() in the Xcpustop assembly code for x86-64
in order to properly acquire a working %gs. This may improve the
handling of panic()s on x86_64.

* Also fix some cases if #if JG'd (ifdef'd out) code in case the
code is ever used later on.

* Protect set_user_TLS() with a critical section to be safe.

* Add debug code to help track down further x86-64 seg-fault issues,
and provide better kprintf()s for the debug path in question.

show more ...

kernel - rewrite the LWKT scheduler's priority mechanism

The purpose of these changes is to begin to address the issue of cpu-bound
kernel threads. For example, the crypto threads, or a HAMMER prun

kernel - rewrite the LWKT scheduler's priority mechanism

The purpose of these changes is to begin to address the issue of cpu-bound
kernel threads. For example, the crypto threads, or a HAMMER prune cycle
that operates entirely out of the buffer cache. These threads tend to hicup
the system, creating temporary lockups because they never switch away due
to their nature as kernel threads.

* Change the LWKT scheduler from a strict hard priority model to
a fair-share with hard priority queueing model.

A kernel thread will be queued with a hard priority, giving it dibs on
the cpu earlier if it has a higher priority. However, if the thread
runs past its fair-share quantum it will then become limited by that
quantum and other lower-priority threads will be allowed to run.

* Rewrite lwkt_yield() and lwkt_user_yield(), remove uio_yield().
Both yield functions are now very fast and can be called without
further timing conditionals, simplifying numerous callers.

lwkt_user_yield() now uses the fair-share quantum to determine when
to yield the cpu for a cpu-bound kernel thread.

* Implement the new yield in the crypto kernel threads, HAMMER, and
other places (many of which already used the old yield functions
which didn't work very well).

* lwkt_switch() now only round-robins after the fair share
quantum is exhausted. It does not necessarily always round robin.

* Separate the critical section count from td_pri. Add td_critcount.

show more ...