[NTOS:KDBG] Reintroduce the capability of KdbpCliInit() to interpret the KDBinit file (#4917)

Addendum to commit baa47fa5e.

Similarly to what was originally done, have KdbpCliInterpretInitFile()
pa

[NTOS:KDBG] Reintroduce the capability of KdbpCliInit() to interpret the KDBinit file (#4917)

Addendum to commit baa47fa5e.

Similarly to what was originally done, have KdbpCliInterpretInitFile()
parse the KDBinit file by breaking back into the debugger.
But contrary to before, replace the deprecated call to KdbEnter() by
a standard DbgBreakPointWithStatus(DBG_STATUS_CONTROL_C) . This allows
KdbEnterDebuggerException() to do the KdbpCliInterpretInitFile() call.

Additional fixes and improvements:

- Run KdbpCliInterpretInitFile() in full KDBG environment (interrupts
disabled, modified IRQL, own stack), like the usual interactive loop.

- The KDBinit data buffer must be in non-paged pool.

- Demote the "Could not open KDBinit" error to a DPRINT, so that it
doesn't pollute the debug log when the KDBG init function is called
early (before the storage stack is initialized), or if the file
doesn't exist -- since this is an optional feature.

show more ...

[NTOS:KD:KDBG] Integration into KD framework (Part 1/3)

Split KdSendPacket and KdReceivePacket into those that manipulate the
KDBG state proper (reside in kdbg/kdbg.c), and those that deal only with

[NTOS:KD:KDBG] Integration into KD framework (Part 1/3)

Split KdSendPacket and KdReceivePacket into those that manipulate the
KDBG state proper (reside in kdbg/kdbg.c), and those that deal only with
debug input/output that will reside in a KDTERM "KD Terminal Driver" DLL.

Based on some previous preparatory work by Hervé Poussineau in PR #4600.
(Equivalents of commits 5162bf106 and partly e9bcf7275.)

show more ...

[NTOS:KDBG] Introduce local versions of KdpPrintString, KdpDprintf and KdpPromptString...

... since the original ones are internal to the kernel and won't be
available once KDBG is moved out of it.

[NTOS:KDBG] Introduce local versions of KdpPrintString, KdpDprintf and KdpPromptString...

... since the original ones are internal to the kernel and won't be
available once KDBG is moved out of it.

Use these functions in the pager/prompt support.

show more ...

[NTOS:KDBG] Isolate terminal and input-related routines from the rest of KDBG. (#5188)

This is done in preparation for moving all this functionality in a
separate KDTERM "KD Terminal Driver" DLL.

A

[NTOS:KDBG] Isolate terminal and input-related routines from the rest of KDBG. (#5188)

This is done in preparation for moving all this functionality in a
separate KDTERM "KD Terminal Driver" DLL.

Additionally:

- Flush the terminal input before sending ANSI escape sequences.

- In KDBG pager, always use the correct reading-key function (the
same used also for reading keys for a line of user input), and not
the simplistic two-call KdbpGetCharSerial + KdbpTryGetCharSerial
that would split the \r \n across calls.

- Call KdbpGetCommandLineSettings() in KdbInitialize() at BootPhase 0,
which is indirectly called by KdDebuggerInitialize0(). And fix its
command-line parsing too.

show more ...

[NTOS:KD:KDBG] Isolate the read-line (prompt) functionality in a separate file.

Rename KdbpReadCommand as KdIoReadLine. Extract the last-command
repetition functionality out of KdIoReadLine and put

[NTOS:KD:KDBG] Isolate the read-line (prompt) functionality in a separate file.

Rename KdbpReadCommand as KdIoReadLine. Extract the last-command
repetition functionality out of KdIoReadLine and put it where it
belongs: only in the KDBG command main loop KdbpCliMainLoop.

show more ...

[NTOS:KDBG] Use KdbpSendCommandSerial() to send specific ANSI escape sequences.

Use this function instead of KdpDprintf(), otherwise, we send them to
**ALL** the display providers, including for exa

[NTOS:KDBG] Use KdbpSendCommandSerial() to send specific ANSI escape sequences.

Use this function instead of KdpDprintf(), otherwise, we send them to
**ALL** the display providers, including for example dmesg. Replaying
the listing with dmesg would then cause the terminal to misbehave later.
For example, it would send the answer of a "Query Device Attributes"
command, as the response to a query for terminal size...

show more ...

[NTOS:KDBG] Move command history management in a separate file. Simplify duplicated code in KdbpReadCommand.

[NTOS:KD] Split internal/kd.h into a part shared in the whole kernel and a private part only used within ntoskrnl/kd.

[NTOS:KD:KDBG] Move some files to a better place.

[NTOS:KDBG] Fix parsing the boot command line for the (NO)LOADSYMBOLS options.

Addendum to commit de892d5b.

The boot options get stripped of their optional command switch '/'
(and replaced by whits

[NTOS:KDBG] Fix parsing the boot command line for the (NO)LOADSYMBOLS options.

Addendum to commit de892d5b.

The boot options get stripped of their optional command switch '/'
(and replaced by whitspace separation) by the NT loader. Also, forbid
the presence of space between the optional '=' character following
(NO)LOADSYMBOLS.

In addition, fix the default initialization of LoadSymbols in KdbSymInit():
we cannot rely on MmNumberOfPhysicalPages in BootPhase 0 since at this point,
the Memory Manager hasn't been initialized and this variable is not yet set.
(We are called by KdInitSystem(0) -> KdDebuggerInitialize0 at kernel init.)
It gets initialized later on between BootPhase 0 and 1.

Also display a nice KDBG signon showing the status of symbols loading.

show more ...

[NTOS:KDBG] Remove dead code. Addendum to baa47fa5e and fe777bb52.

[NTOS:KD][KDBG] Rework the BootPhase >= 2 initialization of the KD/KDBG kernel debugger. (#4892)
CORE-17470

+ KdpDebugLogInit: Add resources cleanup in failure code paths.

Fix, in an NT-compatible

[NTOS:KD][KDBG] Rework the BootPhase >= 2 initialization of the KD/KDBG kernel debugger. (#4892)
CORE-17470

+ KdpDebugLogInit: Add resources cleanup in failure code paths.

Fix, in an NT-compatible manner, how (and when) the KD/KDBG BootPhase >=2
initialization steps are performed.
These are necessary for any functionality KDBG needs, that would depend
on the NT I/O Manager and the storage and filesystem stacks to be running.
This includes, creating the debug log file, and for KDBG, loading its
KDBinit initialization file.

As a result, file debug logging is fixed.

The old ReactOS-specific (NT-incompatible) callback we did in the middle
of IoInitSystem() is removed, in favor of a runtime mechanism that should
work on Windows as well.

The idea for this new mechanism is loosely inspired by the TDL4 rootkit,
see http://blog.w4kfu.com/public/tdl4_article/draft_tdl4article.html
but contrary to it, a specific hook is used instead, as well as the
technique of driver reinitialization:
https://web.archive.org/web/20211021050515/https://driverentry.com.br/en/blog/?p=261

Its rationale is as follows:

We want to be able to perform I/O-related initialization (starting a
logger thread for file log debugging, loading KDBinit file for KDBG,
etc.). A good place for this would be as early as possible, once the
I/O Manager has started the storage and the boot filesystem drivers.

Here is an overview of the initialization steps of the NT Kernel and
Executive:
----
KiSystemStartup(KeLoaderBlock)
if (Cpu == 0) KdInitSystem(0, KeLoaderBlock);
KiSwitchToBootStack() -> KiSystemStartupBootStack()
-> KiInitializeKernel() -> ExpInitializeExecutive(Cpu, KeLoaderBlock)

(NOTE: Any unexpected debugger break will call KdInitSystem(0, NULL); )
KdInitSystem(0, LoaderBlock) -> KdDebuggerInitialize0(LoaderBlock);

ExpInitializeExecutive(Cpu == 0): ExpInitializationPhase = 0;
HalInitSystem(0, KeLoaderBlock); <-- Sets HalInitPnpDriver callback.
...
PsInitSystem(LoaderBlock)
PsCreateSystemThread(Phase1Initialization)

Phase1Initialization(Discard): ExpInitializationPhase = 1;
HalInitSystem(1, KeLoaderBlock);
...
Early initialization of Ob, Ex, Ke.
KdInitSystem(1, KeLoaderBlock);
...
KdDebuggerInitialize1(LoaderBlock);
...
IoInitSystem(LoaderBlock);
...
----
As we can see, KdDebuggerInitialize1() is the last KD initialization
routine the kernel calls, and is called *before* the I/O Manager starts.
Thus, direct Nt/ZwCreateFile ... calls done there would fail. Also,
we want to do the I/O initialization as soon as possible. There does
not seem to be any exported way to be notified about the I/O manager
initialization steps... that is, unless we somehow become a driver and
insert ourselves in the flow!

Since we are not a regular driver, we need to invoke IoCreateDriver()
to create one. However, remember that we are currently running *before*
IoInitSystem(), the I/O subsystem is not initialized yet. Due to this,
calling IoCreateDriver(), much like any other IO functions, would lead
to a crash, because it calls
ObCreateObject(..., IoDriverObjectType, ...), and IoDriverObjectType
is non-initialized yet (it's NULL).

The chosen solution is to hook a "known" exported callback: namely, the
HalInitPnpDriver() callback (it initializes the "HAL Root Bus Driver").
It is set very early on by the HAL via the HalInitSystem(0, ...) call,
and is called early on by IoInitSystem() before any driver is loaded,
but after the I/O Manager has been minimally set up so that new drivers
can be created.
When the hook: KdpInitDriver() is called, we create our driver with
IoCreateDriver(), specifying its entrypoint KdpDriverEntry(), then
restore and call the original HalInitPnpDriver() callback.

Another possible unexplored alternative, could be to insert ourselves
in the KeLoaderBlock->LoadOrderListHead boot modules list, or in the
KeLoaderBlock->BootDriverListHead boot-driver list. (Note that while
we may be able to do this, because boot-drivers are resident in memory,
much like we are, we cannot insert ourselves in the system-driver list
however, since those drivers are expected to come from PE image files.)

Once the KdpDriverEntry() driver entrypoint is called, we register
KdpDriverReinit() for re-initialization with the I/O Manager, in order
to provide more initialization points. KdpDriverReinit() calls the KD
providers at BootPhase >= 2, and schedules further reinitializations
(at most 3 more) if any of the providers request so.

show more ...

[NTOS:KDBG] Split KdbInitialize into KdbSymInit and KDBG initialization proper.

- KdbSymInit() in kdb_symbols.c only initializes symbols implementation
support.
- The rest of KdbInitialize gets mo

[NTOS:KDBG] Split KdbInitialize into KdbSymInit and KDBG initialization proper.

- KdbSymInit() in kdb_symbols.c only initializes symbols implementation
support.
- The rest of KdbInitialize gets moved into kdb_cli.c and initializes
the KDBG debugger itself.
- Move KdbDebugPrint to kdb_cli.c as well.

show more ...

[NTOS:KD] Revisit KdSendPacket() and KdReceivePacket() for DBGKD_DEBUG_IO. (#4914)

- Use SAL2 annotations.
- KdSendPacket(): Validate DEBUG_IO API call.
- KdReceivePacket(): Take the LengthOfStringR

[NTOS:KD] Revisit KdSendPacket() and KdReceivePacket() for DBGKD_DEBUG_IO. (#4914)

- Use SAL2 annotations.
- KdSendPacket(): Validate DEBUG_IO API call.
- KdReceivePacket(): Take the LengthOfStringRead into account; use
KdbpReadCommand() to read the input, so that correct line edition
is available (backspace, etc.)

show more ...

[NTOS:KDBG] Deduplicate code between KdbpPrint() and KdbpPager().

[NTOS:KDBG] Nuke KdbEnter and KdbpCliModuleLoaded

They are not used anymore. Also clean up some obsolete prototypes.

[NTOS:KDBG] Add KdbpPrintUnicodeString

Calling normal unicode functions is not allowed at IRQL > APC_LEVEL, so calling _vsnprintf with unicode parameters from KDBG is invalid.

[KDBG] Fix x64 KDBG

[NTOSKRNL] Regroup the pool allocation tags in one dedicated place

We have a special file, tag.h, which serves as a place to store whatever kernel pool allocation tag yet we still have some tags spa

[NTOSKRNL] Regroup the pool allocation tags in one dedicated place

We have a special file, tag.h, which serves as a place to store whatever kernel pool allocation tag yet we still have some tags sparse over the kernel code... So just re-group them in one unique place.

show more ...

[NTOS:KD][NTOS:KD64][NTOS:KDBG] Fix IRQL & spinlock use

KD64: Raise to HIGH_LEVEL when entering trap
KDBG: lower to DISPATCH_LEVEL when applying IRQL hack & use a worker thread to load symbols
KD&KD

[NTOS:KD][NTOS:KD64][NTOS:KDBG] Fix IRQL & spinlock use

KD64: Raise to HIGH_LEVEL when entering trap
KDBG: lower to DISPATCH_LEVEL when applying IRQL hack & use a worker thread to load symbols
KD&KDBG: Actually unload symbols when required

show more ...

[NTOS:KD] Use a PCONTEXT instead of a PKTRAP_FRAME in KDBG

[NTOS:KD] Move some declarations where they belong

Revert "[NTOS:KDBG] Use CONTEXT instead of KTRAP_FRAME"

This reverts commit e5bffe49da21f5faf2a88a0983ad4876d388e0ea.

CORE-16231

[NTOS:KDBG] Use CONTEXT instead of KTRAP_FRAME

Change KdbpTrapFrameToKdbTrapFrame to prefer CONTEXT (if available) over
KTRAP_FRAME.

[NTOS:KD] Remove some _WINDK_ usages

- Always include kd64.h
- Change KdpPrompt() prototype to be compatible between KDBG and _WINDK_
- Rename KdComponentTable to KdpComponentTable to prevent a conf

[NTOS:KD] Remove some _WINDK_ usages

- Always include kd64.h
- Change KdpPrompt() prototype to be compatible between KDBG and _WINDK_
- Rename KdComponentTable to KdpComponentTable to prevent a conflict
- Add some functions stubs and global variables

show more ...