1.\" Copyright (c) 2003 Greg Lehey 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD: src/share/man/man4/gdb.4,v 1.9 2004/01/28 00:02:10 grog Exp $ 26.\" 27.Dd June 20, 2015 28.Dt GDB 4 29.Os 30.Sh NAME 31.Nm gdb 32.Nd external kernel debugger 33.Sh SYNOPSIS 34.Cd "makeoptions DEBUG=-g" 35.Cd "options DDB" 36.Cd "options GDB_REMOTE_CHAT" 37.Sh DESCRIPTION 38The 39.Nm 40kernel debugger is a variation of 41.Xr gdb 1 42which understands some aspects of the 43.Fx 44kernel environment. 45It can be used in a number of ways: 46.Bl -bullet 47.It 48It can be used to examine the memory of the processor on which it runs. 49.It 50It can be used to analyse a processor dump after a panic. 51.It 52It can be used to debug another system interactively via a serial or firewire 53link. 54In this mode, the processor can be stopped and single stepped. 55.It 56With a firewire link, it can be used to examine the memory of a remote system 57without the participation of that system. 58In this mode, the processor cannot be stopped and single stepped, but it can be 59of use when the remote system has crashed and is no longer responding. 60.El 61.Pp 62When used for remote debugging, 63.Nm 64requires the presence of the 65.Xr ddb 4 66kernel debugger. 67Commands exist to switch between 68.Nm 69and 70.Xr ddb 4 . 71.Sh PREPARING FOR DEBUGGING 72When debugging kernels, it is practically essential to have built a kernel with 73debugging symbols 74.Pq Cd "makeoptions DEBUG=-g" . 75It is easiest to perform operations from the kernel build directory, by default 76.Pa /usr/obj/usr/src/sys/X86_64_GENERIC . 77.Pp 78First, ensure you have a copy of the debug macros in the directory: 79.Pp 80.Dl "make gdbinit" 81.Pp 82This command performs some transformations on the macros installed in 83.Pa /usr/src/tools/debugscripts 84to adapt them to the local environment. 85.Ss "Inspecting the environment of the local machine" 86To look at and change the contents of the memory of the system you are running 87on, 88.Pp 89.Dl "gdb -k -wcore kernel.debug /dev/mem" 90.Pp 91In this mode, you need the 92.Fl k 93flag to indicate to 94.Xr gdb 1 95that the 96.Dq "dump file" 97.Pa /dev/mem 98is a kernel data file. 99You can look at live data, and if you include the 100.Fl wcore 101option, you can change it at your peril. 102The system does not stop (obviously), so a number of things will not work. 103You can set breakpoints, but you cannot 104.Dq continue 105execution, so they will not work. 106.Ss "Debugging a crash dump" 107By default, crash dumps are stored in the directory 108.Pa /var/crash . 109Investigate them from the kernel build directory with: 110.Pp 111.Dl "gdb -k kernel.debug /var/crash/vmcore.29" 112.Pp 113In this mode, the system is obviously stopped, so you can only look at it. 114.Ss "Debugging a live system with a remote link" 115In the following discussion, the term 116.Dq "local system" 117refers to the system running the debugger, and 118.Dq "remote system" 119refers to the live system being debugged. 120.Pp 121To debug a live system with a remote link, the kernel must be compiled with the 122option 123.Cd "options DDB" . 124The option 125.Cd "options BREAK_TO_DEBUGGER" 126enables the debugging machine stop the debugged machine once a connection has 127been established by pressing 128.Ql ^C . 129.Ss "Debugging a live system with a remote serial link" 130When using a serial port for the remote link on the i386 platform, the serial 131port must be identified by setting the flag bit 132.Li 0x80 133for the specified interface. 134Generally, this port will also be used as a serial console (flag bit 135.Li 0x10 ) , 136so the flags field for the interface in the kernel configuration file 137should be: 138.Pp 139.Dl flags 0x90 140.Pp 141To share a console and debug connection on a serial line, use the 142.Cd "options GDB_REMOTE_CHAT" 143option. 144.Ss "Debugging a live system with a remote firewire link" 145As with serial debugging, to debug a live system with a firewire link, the 146kernel must be compiled with the option 147.Cd "options DDB" . 148The 149.Cd "options GDB_REMOTE_CHAT" 150is not necessary, since the firewire implementation uses separate ports for the 151console and debug connection. 152.Pp 153A number of steps must be performed to set up a firewire link: 154.Bl -bullet 155.It 156Ensure that both systems have 157.Xr firewire 4 158support, and that the kernel of the remote system includes the 159.Xr dcons 4 160and 161.Xr dcons_crom 4 162drivers. 163If they are not compiled into the kernel, load the KLDs: 164.Pp 165.Dl "kldload firewire" 166.Pp 167On the remote system only: 168.Bd -literal -offset indent 169kldload dcons 170kldload dcons_crom 171.Ed 172.Pp 173You should see something like this in the 174.Xr dmesg 8 175output of the remote system: 176.Bd -literal -offset indent 177fwohci0: BUS reset 178fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode 179firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 180firewire0: bus manager 1 181firewire0: New S400 device ID:00c04f3226e88061 182dcons_crom0: <dcons configuration ROM> on firewire0 183dcons_crom0: bus_addr 0x22a000 184.Ed 185.Pp 186It is a good idea to load these modules at boot time with the following entry in 187.Pa /boot/loader.conf : 188.Pp 189.Dl dcons_crom_enable="YES" 190.Pp 191This ensures that all three modules are loaded. 192There is no harm in loading 193.Xr dcons 4 194and 195.Xr dcons_crom 4 196on the local system, but if you only want to load the 197.Xr firewire 4 198module, include the following in 199.Pa /boot/loader.conf : 200.Pp 201.Dl firewire_enable="YES" 202.It 203Next, use 204.Xr fwcontrol 8 205to find the firewire node corresponding to the remote machine. 206On the local machine you might see: 207.Bd -literal -offset indent 208# fwcontrol 2092 devices (info_len=2) 210node EUI64 status 211 1 0x00c04f3226e88061 0 212 0 0x000199000003622b 1 213.Ed 214.Pp 215The first node is always the local system, so in this case, node 0 is the remote 216system. 217If there are more than two systems, check from the other end to find which node 218corresponds to the remote system. 219On the remote machine, it looks like this: 220.Bd -literal -offset indent 221# fwcontrol 2222 devices (info_len=2) 223node EUI64 status 224 0 0x000199000003622b 0 225 1 0x00c04f3226e88061 1 226.Ed 227.It 228Next, establish a firewire connection with 229.Xr dconschat 8 : 230.Pp 231.Dl "dconschat -br -G 5556 -t 0x000199000003622b" 232.Pp 233.Li 0x000199000003622b 234is the EUI64 address of the remote node, as determined from the output of 235.Xr fwcontrol 8 236above. 237When started in this manner, 238.Xr dconschat 8 239establishes a local tunnel connection from port 240.Li localhost:5556 241to the remote debugger. 242You can also establish a console port connection with the 243.Fl C 244option to the same invocation 245.Xr dconschat 8 . 246See the 247.Xr dconschat 8 248manpage for further details. 249.Pp 250The 251.Xr dconschat 8 252utility 253does not return control to the user. 254It displays error messages and console output for the remote system, so it is a 255good idea to start it in its own window. 256.It 257Finally, establish connection: 258.Bd -literal -offset indent 259# gdb kernel.debug 260GNU gdb 5.2.1 (FreeBSD) 261.Em "(political statements omitted)" 262Ready to go. Enter 'tr' to connect to the remote target 263with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port 264or 'trf portno' to connect to the remote target with the firewire 265interface. portno defaults to 5556. 266 267Type 'getsyms' after connection to load kld symbols. 268 269If you're debugging a local system, you can use 'kldsyms' instead 270to load the kld symbols. That's a less obnoxious interface. 271(gdb) trf 2720xc21bd378 in ?? () 273.Ed 274.Pp 275The 276.Ic trf 277macro assumes a connection on port 5556. 278If you want to use a different port (by changing the invocation of 279.Xr dconschat 8 280above), use the 281.Ic tr 282macro instead. 283For example, if you want to use port 4711, run 284.Xr dconschat 8 285like this: 286.Pp 287.Dl "dconschat -br -G 4711 -t 0x000199000003622b" 288.Pp 289Then establish connection with: 290.Bd -literal -offset indent 291(gdb) tr localhost:4711 2920xc21bd378 in ?? () 293.Ed 294.El 295.Ss "Non-cooperative debugging a live system with a remote firewire link" 296In addition to the conventional debugging via firewire described in the previous 297section, it is possible to debug a remote system without its cooperation, once 298an initial connection has been established. 299This corresponds to debugging a local machine using 300.Pa /dev/mem . 301It can be very useful if a system crashes and the debugger no longer responds. 302To use this method, set the 303.Xr sysctl 8 304variables 305.Va hw.firewire.fwmem.eui64_hi 306and 307.Va hw.firewire.fwmem.eui64_lo 308to the upper and lower halves of the EUI64 ID of the remote system, 309respectively. 310From the previous example, the remote machine shows: 311.Bd -literal -offset indent 312# fwcontrol 3132 devices (info_len=2) 314node EUI64 status 315 0 0x000199000003622b 0 316 1 0x00c04f3226e88061 1 317.Ed 318.Pp 319Enter: 320.Bd -literal -offset indent 321# sysctl -w hw.firewire.fwmem.eui64_hi=0x00019900 322hw.firewire.fwmem.eui64_hi: 0 -> 104704 323# sysctl -w hw.firewire.fwmem.eui64_lo=0x0003622b 324hw.firewire.fwmem.eui64_lo: 0 -> 221739 325.Ed 326.Pp 327Note that the variables must be explicitly stated in hexadecimal. 328After this, you can examine the remote machine's state with the following input: 329.Bd -literal -offset indent 330# gdb -k kernel.debug /dev/fwmem0.0 331GNU gdb 5.2.1 (FreeBSD) 332.Em "(messages omitted)" 333Reading symbols from /modules/dcons.ko...done. 334Loaded symbols for /modules/dcons.ko 335Reading symbols from /modules/dcons_crom.ko...done. 336Loaded symbols for /modules/dcons_crom.ko 337#0 sched_switch (td=0xc0922fe0) at /usr/src/sys/kern/sched_4bsd.c:621 3380xc21bd378 in ?? () 339.Ed 340.Pp 341In this case, it is not necessary to load the symbols explicitly. 342The remote system continues to run. 343.Sh COMMANDS 344The user interface to 345.Nm 346is via 347.Xr gdb 1 , 348so 349.Xr gdb 1 350commands also work. 351This section discusses only the extensions for kernel debugging that get 352installed in the kernel build directory. 353.Ss "Debugging environment" 354The following macros manipulate the debugging environment: 355.Bl -tag -width indent 356.It Ic ddb 357Switch back to 358.Xr ddb 4 . 359This command is only meaningful when performing remote debugging. 360.It Ic getsyms 361Display 362.Ic kldstat 363information for the target machine and invite user to paste it back in. 364This is required because 365.Nm 366does not allow data to be passed to shell scripts. 367It is necessary for remote debugging and crash dumps; for local memory debugging 368use 369.Ic kldsyms 370instead. 371.It Ic kldsyms 372Read in the symbol tables for the debugging machine. 373This does not work for 374remote debugging and crash dumps; use 375.Ic getsyms 376instead. 377.It Ic tr Ar interface 378Debug a remote system via the specified serial or firewire interface. 379.It Ic tr0 380Debug a remote system via serial interface 381.Pa /dev/cuaa0 . 382.It Ic tr1 383Debug a remote system via serial interface 384.Pa /dev/cuaa1 . 385.It Ic trf 386Debug a remote system via firewire interface at default port 5556. 387.El 388.Pp 389The commands 390.Ic tr0 , tr1 391and 392.Ic trf 393are convenience commands which invoke 394.Ic tr . 395.Ss "The current process environment" 396The following macros are convenience functions intended to make things easier 397than the standard 398.Xr gdb 1 399commands. 400.Bl -tag -width indent 401.It Ic f0 402Select stack frame 0 and show assembler-level details. 403.It Ic f1 404Select stack frame 1 and show assembler-level details. 405.It Ic f2 406Select stack frame 2 and show assembler-level details. 407.It Ic f3 408Select stack frame 3 and show assembler-level details. 409.It Ic f4 410Select stack frame 4 and show assembler-level details. 411.It Ic f5 412Select stack frame 5 and show assembler-level details. 413.It Ic xb 414Show 12 words in hex, starting at current 415.Va ebp 416value. 417.It Ic xi 418List the next 10 instructions from the current 419.Va eip 420value. 421.It Ic xp 422Show the register contents and the first four parameters of the current stack 423frame. 424.It Ic xp0 425Show the first parameter of current stack frame in various formats. 426.It Ic xp1 427Show the second parameter of current stack frame in various formats. 428.It Ic xp2 429Show the third parameter of current stack frame in various formats. 430.It Ic xp3 431Show the fourth parameter of current stack frame in various formats. 432.It Ic xp4 433Show the fifth parameter of current stack frame in various formats. 434.It Ic xs 435Show the last 12 words on stack in hexadecimal. 436.It Ic xxp 437Show the register contents and the first ten parameters. 438.It Ic z 439Single step 1 instruction (over calls) and show next instruction. 440.It Ic zs 441Single step 1 instruction (through calls) and show next instruction. 442.El 443.Ss "Examining other processes" 444The following macros access other processes. 445The 446.Nm 447debugger 448does not understand the concept of multiple processes, so they effectively 449bypass the entire 450.Nm 451environment. 452.Bl -tag -width indent 453.It Ic btp Ar pid 454Show a backtrace for the process 455.Ar pid . 456.It Ic btpa 457Show backtraces for all processes in the system. 458.It Ic btpp 459Show a backtrace for the process previously selected with 460.Ic defproc . 461.It Ic btr Ar ebp 462Show a backtrace from the 463.Ar ebp 464address specified. 465.It Ic defproc Ar pid 466Specify the PID of the process for some other commands in this section. 467.It Ic fr Ar frame 468Show frame 469.Ar frame 470of the stack of the process previously selected with 471.Ic defproc . 472.It Ic pcb Ar proc 473Show some PCB contents of the process 474.Ar proc . 475.El 476.Ss "Examining data structures" 477You can use standard 478.Xr gdb 1 479commands to look at most data structures. 480The macros in this section are 481convenience functions which typically display the data in a more readable 482format, or which omit less interesting parts of the structure. 483.Bl -tag -width indent 484.It Ic bp 485Show information about the buffer header pointed to by the variable 486.Va bp 487in the current frame. 488.It Ic bpd 489Show the contents 490.Pq Vt "char *" 491of 492.Va bp->data 493in the current frame. 494.It Ic bpl 495Show detailed information about the buffer header 496.Pq Vt "struct bp" 497pointed at by the local variable 498.Va bp . 499.It Ic bpp Ar bp 500Show summary information about the buffer header 501.Pq Vt "struct bp" 502pointed at by the parameter 503.Ar bp . 504.It Ic bx 505Print a number of fields from the buffer header pointed at in by the pointer 506.Ar bp 507in the current environment. 508.It Ic vdev 509Show some information of the 510.Vt vnode 511pointed to by the local variable 512.Va vp . 513.El 514.Ss "Miscellaneous macros" 515.Bl -tag -width indent 516.It Ic checkmem 517Check unallocated memory for modifications. 518This assumes that the kernel has been compiled with 519.Cd "options DIAGNOSTIC" 520This causes the contents of free memory to be set to 521.Li 0xdeadc0de . 522.It Ic dmesg 523Print the system message buffer. 524This corresponds to the 525.Xr dmesg 8 526utility. 527This macro used to be called 528.Ic msgbuf . 529It can take a very long time over a serial line, 530and it is even slower via firewire 531or local memory due to inefficiencies in 532.Nm . 533When debugging a crash dump or over firewire, it is not necessary to start 534.Nm 535to access the message buffer: instead, use an appropriate variation of 536.Bd -literal -offset indent 537dmesg -M /var/crash/vmcore.0 -N kernel.debug 538dmesg -M /dev/fwmem0.0 -N kernel.debug 539.Ed 540.It Ic kldstat 541Equivalent of the 542.Xr kldstat 8 543utility without options. 544.It Ic pname 545Print the command name of the current process. 546.It Ic ps 547Show process status. 548This corresponds in concept, but not in appearance, to the 549.Xr ps 1 550utility. 551When debugging a crash dump or over firewire, it is not necessary to start 552.Nm 553to display the 554.Xr ps 1 555output: instead, use an appropriate variation of 556.Bd -literal -offset indent 557ps -M /var/crash/vmcore.0 -N kernel.debug 558ps -M /dev/fwmem0.0 -N kernel.debug 559.Ed 560.It Ic y 561Kludge for writing macros. 562When writing macros, it is convenient to paste them 563back into the 564.Nm 565window. 566Unfortunately, if the macro is already defined, 567.Nm 568insists on asking 569.Pp 570.Dl "Redefine foo?" 571.Pp 572It will not give up until you answer 573.Ql y . 574This command is that answer. 575It does nothing else except to print a warning 576message to remind you to remove it again. 577.El 578.Sh SEE ALSO 579.Xr gdb 1 , 580.Xr ps 1 , 581.Xr ddb 4 , 582.Xr firewire 4 , 583.Xr vinumdebug 4 , 584.Xr dconschat 8 , 585.Xr dmesg 8 , 586.Xr fwcontrol 8 , 587.Xr kldload 8 588.Sh AUTHORS 589This man page was written by 590.An Greg Lehey Aq Mt grog@FreeBSD.org . 591.Sh BUGS 592The 593.Xr gdb 1 594debugger 595was never designed to debug kernels, and it is not a very good match. 596Many problems exist. 597.Pp 598The 599.Nm 600implementation is very inefficient, and many operations are slow. 601.Pp 602Serial debugging is even slower, and race conditions can make it difficult to 603run the link at more than 9600 bps. 604Firewire connections do not have this problem. 605.Pp 606The debugging macros 607.Dq "just growed" . 608In general, the person who wrote them did so while looking for a specific 609problem, so they may not be general enough, and they may behave badly when used 610in ways for which they were not intended, even if those ways make sense. 611.Pp 612Many of these commands only work on the ia32 architecture. 613