kernel: Remove numerous #include <sys/thread2.h>.

Most of them were added when we converted spl*() calls to
crit_enter()/crit_exit(), almost 14 years ago. We can now
remove a good chunk of them agai

kernel: Remove numerous #include <sys/thread2.h>.

Most of them were added when we converted spl*() calls to
crit_enter()/crit_exit(), almost 14 years ago. We can now
remove a good chunk of them again for where crit_*() are
no longer used.

I had to adjust some files that were relying on thread2.h
or headers that it includes coming in via other headers
that it was removed from.

show more ...

kernel: Move us to using M_NOWAIT and M_WAITOK for mbuf functions.

The main reason is that our having to use the MB_WAIT and MB_DONTWAIT
flags was a recurring issue when porting drivers from FreeBSD

kernel: Move us to using M_NOWAIT and M_WAITOK for mbuf functions.

The main reason is that our having to use the MB_WAIT and MB_DONTWAIT
flags was a recurring issue when porting drivers from FreeBSD because
it tended to get forgotten and the code would compile anyway with the
wrong constants. And since MB_WAIT and MB_DONTWAIT ended up as ocflags
for an objcache_get() or objcache_reclaimlist call (which use M_WAITOK
and M_NOWAIT), it was just one big converting back and forth with some
sanitization in between.

This commit allows M_* again for the mbuf functions and keeps the
sanitizing as it was before: when M_WAITOK is among the passed flags,
objcache functions will be called with M_WAITOK and when it is absent,
they will be called with M_NOWAIT. All other flags are scrubbed by the
MB_OCFLAG() macro which does the same as the former MBTOM().

Approved-by: dillon

show more ...

kernel: make pktinfo and cpuid native to ip_input()

In order to remove ether_input_pkt(), switch the prototype
of if_input() and adjust all callers. While there, consolidate
the style of the invoke

kernel: make pktinfo and cpuid native to ip_input()

In order to remove ether_input_pkt(), switch the prototype
of if_input() and adjust all callers. While there, consolidate
the style of the invoke.

Suggested and reviewed by: sephe

show more ...

ifq: Remove the unused parameter 'mpolled' from ifq dequeue interface

The ifq_poll() -> ifq_dequeue() model is not MPSAFE, and mpolled has
not been used, i.e. set to NULL, for years; time to let it

ifq: Remove the unused parameter 'mpolled' from ifq dequeue interface

The ifq_poll() -> ifq_dequeue() model is not MPSAFE, and mpolled has
not been used, i.e. set to NULL, for years; time to let it go.

show more ...

kernel: Use DEVMETHOD_END in the drivers.

if: Per-cpu ifnet/ifaddr statistics, step 1/3

Wrap ifnet/ifaddr stats updating, setting and extraction into macros;
ease upcoming changes.

if: Multiple TX queue support step 1 of many; introduce ifaltq subqueue

Put the plain queue information, e.g. queue header and tail, serializer,
packet staging scoreboard and ifnet.if_start schedule

if: Multiple TX queue support step 1 of many; introduce ifaltq subqueue

Put the plain queue information, e.g. queue header and tail, serializer,
packet staging scoreboard and ifnet.if_start schedule netmsg etc. into
its own structure (subqueue). ifaltq structure could have multiple of
subqueues based on the count that drivers can specify.

Subqueue's enqueue, dequeue, purging and states updating are protected
by the subqueue's serializer, so for hardwares supporting multiple TX
queues, contention on queuing operation could be greatly reduced.

The subqueue is passed to if_start to let the driver know which hardware
TX queue to work on. Only the related driver's TX queue serializer will
be held, so for hardwares supporting multiple TX queues, contention on
driver's TX queue serializer could be greatly reduced.

Bunch of ifsq_ prefixed functions are added, which is used to perform
various operations on subqueues. Commonly used ifq_ prefixed functions
are still kept mainly for the drivers which do not support multiple TX
queues (well, these functions also ease the netif/ convertion in this
step :).

All of the pseudo network devices under sys/net are converted to use the
new subqueue operation. netproto/802_11 is converted too. igb(4) is
converted to use the new subqueue operation, the rest of the network
drivers are only changed for the if_start interface modification.

For ALTQs which have packet scheduler enabled, only the first subqueue
is used (*).

(*) Whether we should utilize multiple TX queues if ALTQ's packet scheduler
is enabled is quite questionable. Mainly because hardware's multiple TX
queue packet dequeue mechanism could have negative impact on ALTQ's packet
scheduler's decision.

show more ...

if: Move if_cpuid into ifaltq; prepare multiple TX queues support

if_cpuid and if_npoll_cpuid are merged and moved into ifaltq as
altq_cpuid, which indicates the owner CPU of the tx queue. Since
we

if: Move if_cpuid into ifaltq; prepare multiple TX queues support

if_cpuid and if_npoll_cpuid are merged and moved into ifaltq as
altq_cpuid, which indicates the owner CPU of the tx queue. Since
we already have code in if_start_dispatch() to catching tx queue
owner CPU changes, this merging is quite safe.

show more ...

kernel: Remove some unused variables in network drivers.

if: Move IFF_OACTIVE bit into ifaltq; prepare multiple TX queues support

ifaltq.altq_hw_oactive is now used to record that NIC's TX queue is full.
IFF_OACTIVE is removed from kernel. User space IFF

if: Move IFF_OACTIVE bit into ifaltq; prepare multiple TX queues support

ifaltq.altq_hw_oactive is now used to record that NIC's TX queue is full.
IFF_OACTIVE is removed from kernel. User space IFF_OACTIVE is kept for
compability.

ifaltq.altq_hw_oactive should not be accessed directly. Following set of
functions are provided and should be used:
ifq_is_oactive(ifnet.if_snd) - Whether NIC's TX queue is full or not
ifq_set_oactive(ifnet.if_snd) - NIC's TX queue is full
ifq_clr_oactive(ifnet.if_snd) - NIC's TX queue is no longer full

show more ...

fwe: Switch from device_polling to ifpoll

Initial import of binutils 2.22 on the new vendor branch

Future versions of binutils will also reside on this branch rather
than continuing to create new binutils branches for each new version.

kernel: Use NULL for DRIVER_MODULE()'s evh & arg (which are pointers).

This is just cosmetics for easier reading.

Reduce ifnet.if_serializer contention on output path:
- Push ifnet.if_serializer holding down into each ifnet.if_output implementation
- Add a serializer into ifaltq, which is used to protect send qu

Reduce ifnet.if_serializer contention on output path:
- Push ifnet.if_serializer holding down into each ifnet.if_output implementation
- Add a serializer into ifaltq, which is used to protect send queue instead of
its parent's if_serializer. This change has following implication:
o On output path, enqueueing packets and calling ifnet.if_start are decoupled
o In device drivers, poll->dev_encap_ok->dequeue operation sequence is no
longer safe, instead dequeue->dev_encap_fail->prepend should be used
This serializer will be held by using lwkt_serialize_adaptive_enter()
- Add altq_started field into ifaltq, which is used to interlock the calling
of its parent's if_start, to reduce ifnet.if_serializer contention.
if_devstart(), a helper function which utilizes ifaltq.altq_started, is added
to reduce code duplication in ethernet device drivers.
- Add if_cpuid into ifnet. This field indicates on which CPU device driver's
interrupt will happen.
- Add ifq_dispatch(). This function will try to hold ifnet.if_serializer in
order to call ifnet.if_start. If this attempt fails, this function will
schedule ifnet.if_start to be called on CPU located by ifnet.if_start_cpuid
if_start_nmsg, which is per-CPU netmsg, is added to ifnet to facilitate
ifnet.if_start scheduling. ifq_dispatch() is called by ether_output_frame()
currently
- Use ifq_classic_ functions, if altq is not enabled
- Fix various device drivers bugs in their if_start implementation
- Add ktr for ifq classic enqueue and dequeue
- Add ktr for ifnet.if_start

show more ...

Remove bogus checks after kmalloc(M_WAITOK) which never returns NULL.

Reviewed-by: hasso

- Make fwe(4) aware of IFF_POLLING setting in its if_init().
- Allow IFF_POLLING flag to be turned on even when the interface is not
up yet, since after above fix, all drivers that support polling(

- Make fwe(4) aware of IFF_POLLING setting in its if_init().
- Allow IFF_POLLING flag to be turned on even when the interface is not
up yet, since after above fix, all drivers that support polling(4) will
turn on/off interrupt in their if_init() based on IFF_POLLING.

This fixes the bug reported by many people that enabling "polling" in
rc.conf's ifconfig_ifaceX does not work.

show more ...

Rename sprintf -> ksprintf
Rename snprintf -> knsprintf

Make allowances for source files that are compiled for both userland and
the kernel.

Do a major clean-up of the BUSDMA architecture. A large number of
essentially machine-independant drivers use the structures and definitions
in machine-dependant directories that are really machine-

Do a major clean-up of the BUSDMA architecture. A large number of
essentially machine-independant drivers use the structures and definitions
in machine-dependant directories that are really machine-independant in
nature.

Split <machine/bus_dma.h> into machine-depdendant and machine-independant parts
and make the primary access run through <sys/bus_dma.h>.

Remove <machine/bus.h>, <machine/bus_memio.h> and <machine/bus_pio.h>. The
optimizations related to bus_memio.h and bus_pio.h made a huge mess,
introduced machine-specific knowledge into essentially machine-independant
drivers, and required specific #include file orderings to do their job.
They may be reintroduced in some other form later on.

Move <machine/resource.h> to <sys/bus_resource.h>. The contents of the file
is machine-independant or can be made a superset across many platforms.

Make <sys/bus.h> include <sys/bus_dma.h> and <sys/bus_resource.h> and
include <sys/bus.h> where necessary. Remove all #include's of
<machine/resource.h> and <machine/bus.h>. That is, make the BUSDMA
infrastructure integral to I/O-mapped and memory-mapped accesses to devices
and remove a large chunk of machine-specific dependancies from drivers.
bus_if.h and device_if.h are now required to be present when using <sys/bus.h>.

show more ...

Rename malloc->kmalloc, free->kfree, and realloc->krealloc. Pass 2

Rename malloc->kmalloc, free->kfree, and realloc->krealloc. Pass 1

{ether,ieee80211}_ifdetach() can't be called with serializer being held, since
they will go through code which tries to hold serializer again, e.g.
ether_ifdetach() -> if_detach() -> in_control()

So

{ether,ieee80211}_ifdetach() can't be called with serializer being held, since
they will go through code which tries to hold serializer again, e.g.
ether_ifdetach() -> if_detach() -> in_control()

So in various NICs' xxx_detach():
- Move bus_teardown_intr() under "(device_is_attached())", whenever it is
applicable. Since it is not possible that intrhandle is NULL here, nuke
original "(intrhandle != NULL)". This can:
1) Avoid holding serializer, if xxx_attach() fails
2) Release serializer ASAP
3) Ease following tasks
- Hold serializer only for xxx_stop()(or similar functions which stops NIC) and
bus_teardown_intr()
- Call {ether,ieee80211}_ifdetach() after serializer is released

Other stuffs:
- Serialize xxx_detach() for awi(4), ep(4), sn(4) and xe(4)
- Release serializer before returning from {ed_pccard,ray}_detach()
- Make ipw(4)'s ipw_detach() suitable for error handling, adjust ipw_attach()
accordingly
- Fix a bug in ex_pccard_detach(): instead of if_detach(), ether_ifdetach()
should be used here
- For ndis(4), "ifp->serializer" ==> "ifp->if_serializer"
Reported-by: esmith <esmith@postmark.net>

Discussed-with: dillon and joerg
Partially-Reviewed-by: dillon and joerg
Reported-by: Steve Mynott <steve.mynott@gmail.com> and me

show more ...

Make all network interrupt service routines MPSAFE part 1/3.

Replace the critical section that was previously used to serialize access
with the LWKT serializer. Integrate the serializer into the IF

Make all network interrupt service routines MPSAFE part 1/3.

Replace the critical section that was previously used to serialize access
with the LWKT serializer. Integrate the serializer into the IFNET structure.
Note that kern.intr_mpsafe must be set to 1 for network interrupts to actually
run MPSAFE. Also note that any interrupts shared with othre non-MP drivers
will cause all drivers on that interrupt to run with the Big Giant Lock.

Network interrupt - Each network driver then simply passes that serializer
to bus_setup_intr() so only a single serializer is required to process the
entire interrupt path. LWKT serialization support is already 100% integrated
into the interrupt subsystem so it will already be held as of when the
registered interrupt procedure is called.

Ioctl and if_* functions - All callers of if_* functions (such as if_start,
if_ioctl, etc) now obtain the IFNET serializer before making the call. Thus
all of these entry points into the driver will now be serialized.

if_input - All code that calls if_input now ensures that the serializer
is held. It will either already be held (when called from a driver), or
the serializer will be wrapped around the call. When packets are forwarded
or bridged between interfaces, the target interface serializer will be
dropped temporarily to avoid a deadlock.

Device Driver access - dev_* entry points into certain pseudo-network
devices now obtain and release the serializer. This had to be done on
a device-by-device basis (but there are only a few such devices).

Thanks to several people for helping test the patch, in particular
Sepherosa Ziehau.

show more ...

Fix the design of ifq_dequeue/altq_dequeue by adding an mbuf pointer and
requiring that a polled mbuf be passed as an argument to the dequeue
function. Assert that the passed argument matches the mb

Fix the design of ifq_dequeue/altq_dequeue by adding an mbuf pointer and
requiring that a polled mbuf be passed as an argument to the dequeue
function. Assert that the passed argument matches the mbuf that is actually
dequeued. Also remove assignments of the return value from ifq_dequeue()
in such cases which implied that the mbuf might be different when, in fact,
it had better not be.

show more ...

Cleanup some of the newbus infrastructure.

* Change the device_identify API to return success/failure, like
most of the other newbus methods. This may be used for conflict
resolution in the fut

Cleanup some of the newbus infrastructure.

* Change the device_identify API to return success/failure, like
most of the other newbus methods. This may be used for conflict
resolution in the future.

* Clearly document the device_identify method and formalize its use
by adding discrimination between initial bus probes and bus rescans.
Do not re-execute static identification code that has already been run
every time a new driver is added at run-time.

* Clearly document the do-ISA-last hack.

* Provide generic routines for the most common device_identify operations
(psueo or synthesized devices that operate under other devices, such as
lpt operating under ppbus, which are not 'scanned' by the parent bus).

* Remove the hacks that install and initialize the nexus device. Instead,
use the existing DRIVER_MODULE infrastructure to install nexus under
root_bus.

* Document the boot-time initialization path so it doesn't take the next
guy 8 hours to figure out what code is actually being run when.

show more ...

- Move DEVICE_POLLING from opt_global.h to opt_polling.h(newly added),
so that polling(4) can be enabled in modules that are not built
during kernel building
- Add opt_polling.h to files that dep

- Move DEVICE_POLLING from opt_global.h to opt_polling.h(newly added),
so that polling(4) can be enabled in modules that are not built
during kernel building
- Add opt_polling.h to files that depend on DEVICE_POLLING
- Change related netif modules' Makefile to enable polling(4) support
- Add comment in net/if_var.h to prevent DEVICE_POLLING related
incompatibilities from being introduced
Suggested-by: dillon

NOTE: As of this commit, any file that will depend on DEVICE_POLLING
*must* include opt_polling.h at its beginning

With-helps-from: joerg
Reviewed-by: dillon, submit@

show more ...