1.\" Copyright (c) 2003 Greg Lehey 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD: src/share/man/man4/gdb.4,v 1.9 2004/01/28 00:02:10 grog Exp $ 26.\" $DragonFly: src/share/man/man4/gdb.4,v 1.2 2004/12/20 14:21:19 swildner Exp $ 27.\" 28.Dd December 30, 2003 29.Dt GDB 4 30.Os 31.Sh NAME 32.Nm gdb 33.Nd external kernel debugger 34.Sh SYNOPSIS 35.Cd "makeoptions DEBUG=-g" 36.Cd "options DDB" 37.Cd "options GDB_REMOTE_CHAT" 38.Sh DESCRIPTION 39The 40.Nm 41kernel debugger is a variation of 42.Xr gdb 1 43which understands some aspects of the 44.Fx 45kernel environment. 46It can be used in a number of ways: 47.Bl -bullet 48.It 49It can be used to examine the memory of the processor on which it runs. 50.It 51It can be used to analyse a processor dump after a panic. 52.It 53It can be used to debug another system interactively via a serial or firewire 54link. 55In this mode, the processor can be stopped and single stepped. 56.It 57With a firewire link, it can be used to examine the memory of a remote system 58without the participation of that system. 59In this mode, the processor cannot be stopped and single stepped, but it can be 60of use when the remote system has crashed and is no longer responding. 61.El 62.Pp 63When used for remote debugging, 64.Nm 65requires the presence of the 66.Xr ddb 4 67kernel debugger. 68Commands exist to switch between 69.Nm 70and 71.Xr ddb 4 . 72.Sh PREPARING FOR DEBUGGING 73When debugging kernels, it is practically essential to have built a kernel with 74debugging symbols 75.Pq Cd "makeoptions DEBUG=-g" . 76It is easiest to perform operations from the kernel build directory, by default 77.Pa /usr/obj/usr/src/sys/GENERIC . 78.Pp 79First, ensure you have a copy of the debug macros in the directory: 80.Pp 81.Dl "make gdbinit" 82.Pp 83This command performs some transformations on the macros installed in 84.Pa /usr/src/tools/debugscripts 85to adapt them to the local environment. 86.Ss "Inspecting the environment of the local machine" 87To look at and change the contents of the memory of the system you are running 88on, 89.Pp 90.Dl "gdb -k -wcore kernel.debug /dev/mem" 91.Pp 92In this mode, you need the 93.Fl k 94flag to indicate to 95.Xr gdb 1 96that the 97.Dq "dump file" 98.Pa /dev/mem 99is a kernel data file. 100You can look at live data, and if you include the 101.Fl wcore 102option, you can change it at your peril. 103The system does not stop (obviously), so a number of things will not work. 104You can set breakpoints, but you cannot 105.Dq continue 106execution, so they will not work. 107.Ss "Debugging a crash dump" 108By default, crash dumps are stored in the directory 109.Pa /var/crash . 110Investigate them from the kernel build directory with: 111.Pp 112.Dl "gdb -k kernel.debug /var/crash/vmcore.29" 113.Pp 114In this mode, the system is obviously stopped, so you can only look at it. 115.Ss "Debugging a live system with a remote link" 116In the following discussion, the term 117.Dq "local system" 118refers to the system running the debugger, and 119.Dq "remote system" 120refers to the live system being debugged. 121.Pp 122To debug a live system with a remote link, the kernel must be compiled with the 123option 124.Cd "options DDB" . 125The option 126.Cd "options BREAK_TO_DEBUGGER" 127enables the debugging machine stop the debugged machine once a connection has 128been established by pressing 129.Ql ^C . 130.Ss "Debugging a live system with a remote serial link" 131When using a serial port for the remote link on the i386 platform, the serial 132port must be identified by setting the flag bit 133.Li 0x80 134for the specified interface. 135Generally, this port will also be used as a serial console (flag bit 136.Li 0x10 ) , 137so the entry in 138.Pa /boot/device.hints 139should be: 140.Pp 141.Dl hint.sio.0.flags="0x90" 142.Pp 143To share a console and debug connection on a serial line, use the 144.Cd "options GDB_REMOTE_CHAT" 145option. 146.Ss "Debugging a live system with a remote firewire link" 147As with serial debugging, to debug a live system with a firewire link, the 148kernel must be compiled with the option 149.Cd "options DDB" . 150The 151.Cd "options GDB_REMOTE_CHAT" 152is not necessary, since the firewire implementation uses separate ports for the 153console and debug connection. 154.Pp 155A number of steps must be performed to set up a firewire link: 156.Bl -bullet 157.It 158Ensure that both systems have 159.Xr firewire 4 160support, and that the kernel of the remote system includes the 161.Xr dcons 4 162and 163.Xr dcons_crom 4 164drivers. 165If they are not compiled into the kernel, load the KLDs: 166.Pp 167.Dl "kldload firewire" 168.Pp 169On the remote system only: 170.Bd -literal -offset indent 171kldload dcons 172kldload dcons_crom 173.Ed 174.Pp 175You should see something like this in the 176.Xr dmesg 8 177output of the remote system: 178.Bd -literal -offset indent 179fwohci0: BUS reset 180fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode 181firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 182firewire0: bus manager 1 183firewire0: New S400 device ID:00c04f3226e88061 184dcons_crom0: <dcons configuration ROM> on firewire0 185dcons_crom0: bus_addr 0x22a000 186.Ed 187.Pp 188It is a good idea to load these modules at boot time with the following entry in 189.Pa /boot/loader.conf : 190.Pp 191.Dl dcons_crom_enable="YES" 192.Pp 193This ensures that all three modules are loaded. 194There is no harm in loading 195.Xr dcons 4 196and 197.Xr dcons_crom 4 198on the local system, but if you only want to load the 199.Xr firewire 4 200module, include the following in 201.Pa /boot/loader.conf : 202.Pp 203.Dl firewire_enable="YES" 204.It 205Next, use 206.Xr fwcontrol 8 207to find the firewire node corresponding to the remote machine. 208On the local machine you might see: 209.Bd -literal -offset indent 210# fwcontrol 2112 devices (info_len=2) 212node EUI64 status 213 1 0x00c04f3226e88061 0 214 0 0x000199000003622b 1 215.Ed 216.Pp 217The first node is always the local system, so in this case, node 0 is the remote 218system. 219If there are more than two systems, check from the other end to find which node 220corresponds to the remote system. 221On the remote machine, it looks like this: 222.Bd -literal -offset indent 223# fwcontrol 2242 devices (info_len=2) 225node EUI64 status 226 0 0x000199000003622b 0 227 1 0x00c04f3226e88061 1 228.Ed 229.It 230Next, establish a firewire connection with 231.Xr dconschat 8 : 232.Pp 233.Dl "dconschat -br -G 5556 -t 0x000199000003622b" 234.Pp 235.Li 0x000199000003622b 236is the EUI64 address of the remote node, as determined from the output of 237.Xr fwcontrol 8 238above. 239When started in this manner, 240.Xr dconschat 8 241establishes a local tunnel connection from port 242.Li localhost:5556 243to the remote debugger. 244You can also establish a console port connection with the 245.Fl C 246option to the same invocation 247.Xr dconschat 8 . 248See the 249.Xr dconschat 8 250manpage for further details. 251.Pp 252The 253.Xr dconschat 8 254utility 255does not return control to the user. 256It displays error messages and console output for the remote system, so it is a 257good idea to start it in its own window. 258.It 259Finally, establish connection: 260.Bd -literal -offset indent 261# gdb kernel.debug 262GNU gdb 5.2.1 (FreeBSD) 263.Em "(political statements omitted)" 264Ready to go. Enter 'tr' to connect to the remote target 265with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port 266or 'trf portno' to connect to the remote target with the firewire 267interface. portno defaults to 5556. 268 269Type 'getsyms' after connection to load kld symbols. 270 271If you're debugging a local system, you can use 'kldsyms' instead 272to load the kld symbols. That's a less obnoxious interface. 273(gdb) trf 2740xc21bd378 in ?? () 275.Ed 276.Pp 277The 278.Ic trf 279macro assumes a connection on port 5556. 280If you want to use a different port (by changing the invocation of 281.Xr dconschat 8 282above), use the 283.Ic tr 284macro instead. 285For example, if you want to use port 4711, run 286.Xr dconschat 8 287like this: 288.Pp 289.Dl "dconschat -br -G 4711 -t 0x000199000003622b" 290.Pp 291Then establish connection with: 292.Bd -literal -offset indent 293(gdb) tr localhost:4711 2940xc21bd378 in ?? () 295.Ed 296.El 297.Ss "Non-cooperative debugging a live system with a remote firewire link" 298In addition to the conventional debugging via firewire described in the previous 299section, it is possible to debug a remote system without its cooperation, once 300an initial connection has been established. 301This corresponds to debugging a local machine using 302.Pa /dev/mem . 303It can be very useful if a system crashes and the debugger no longer responds. 304To use this method, set the 305.Xr sysctl 8 306variables 307.Va hw.firewire.fwmem.eui64_hi 308and 309.Va hw.firewire.fwmem.eui64_lo 310to the upper and lower halves of the EUI64 ID of the remote system, 311respectively. 312From the previous example, the remote machine shows: 313.Bd -literal -offset indent 314# fwcontrol 3152 devices (info_len=2) 316node EUI64 status 317 0 0x000199000003622b 0 318 1 0x00c04f3226e88061 1 319.Ed 320.Pp 321Enter: 322.Bd -literal -offset indent 323# sysctl -w hw.firewire.fwmem.eui64_hi=0x00019900 324hw.firewire.fwmem.eui64_hi: 0 -> 104704 325# sysctl -w hw.firewire.fwmem.eui64_lo=0x0003622b 326hw.firewire.fwmem.eui64_lo: 0 -> 221739 327.Ed 328.Pp 329Note that the variables must be explicitly stated in hexadecimal. 330After this, you can examine the remote machine's state with the following input: 331.Bd -literal -offset indent 332# gdb -k kernel.debug /dev/fwmem0.0 333GNU gdb 5.2.1 (FreeBSD) 334.Em "(messages omitted)" 335Reading symbols from /modules/dcons.ko...done. 336Loaded symbols for /modules/dcons.ko 337Reading symbols from /modules/dcons_crom.ko...done. 338Loaded symbols for /modules/dcons_crom.ko 339#0 sched_switch (td=0xc0922fe0) at /usr/src/sys/kern/sched_4bsd.c:621 3400xc21bd378 in ?? () 341.Ed 342.Pp 343In this case, it is not necessary to load the symbols explicitly. 344The remote system continues to run. 345.Sh COMMANDS 346The user interface to 347.Nm 348is via 349.Xr gdb 1 , 350so 351.Xr gdb 1 352commands also work. 353This section discusses only the extensions for kernel debugging that get 354installed in the kernel build directory. 355.Ss "Debugging environment" 356The following macros manipulate the debugging environment: 357.Bl -tag -width indent 358.It Ic ddb 359Switch back to 360.Xr ddb 4 . 361This command is only meaningful when performing remote debugging. 362.It Ic getsyms 363Display 364.Ic kldstat 365information for the target machine and invite user to paste it back in. 366This is required because 367.Nm 368does not allow data to be passed to shell scripts. 369It is necessary for remote debugging and crash dumps; for local memory debugging 370use 371.Ic kldsyms 372instead. 373.It Ic kldsyms 374Read in the symbol tables for the debugging machine. 375This does not work for 376remote debugging and crash dumps; use 377.Ic getsyms 378instead. 379.It Ic tr Ar interface 380Debug a remote system via the specified serial or firewire interface. 381.It Ic tr0 382Debug a remote system via serial interface 383.Pa /dev/cuaa0 . 384.It Ic tr1 385Debug a remote system via serial interface 386.Pa /dev/cuaa1 . 387.It Ic trf 388Debug a remote system via firewire interface at default port 5556. 389.El 390.Pp 391The commands 392.Ic tr0 , tr1 393and 394.Ic trf 395are convenience commands which invoke 396.Ic tr . 397.Ss "The current process environment" 398The following macros are convenience functions intended to make things easier 399than the standard 400.Xr gdb 1 401commands. 402.Bl -tag -width indent 403.It Ic f0 404Select stack frame 0 and show assembler-level details. 405.It Ic f1 406Select stack frame 1 and show assembler-level details. 407.It Ic f2 408Select stack frame 2 and show assembler-level details. 409.It Ic f3 410Select stack frame 3 and show assembler-level details. 411.It Ic f4 412Select stack frame 4 and show assembler-level details. 413.It Ic f5 414Select stack frame 5 and show assembler-level details. 415.It Ic xb 416Show 12 words in hex, starting at current 417.Va ebp 418value. 419.It Ic xi 420List the next 10 instructions from the current 421.Va eip 422value. 423.It Ic xp 424Show the register contents and the first four parameters of the current stack 425frame. 426.It Ic xp0 427Show the first parameter of current stack frame in various formats. 428.It Ic xp1 429Show the second parameter of current stack frame in various formats. 430.It Ic xp2 431Show the third parameter of current stack frame in various formats. 432.It Ic xp3 433Show the fourth parameter of current stack frame in various formats. 434.It Ic xp4 435Show the fifth parameter of current stack frame in various formats. 436.It Ic xs 437Show the last 12 words on stack in hexadecimal. 438.It Ic xxp 439Show the register contents and the first ten parameters. 440.It Ic z 441Single step 1 instruction (over calls) and show next instruction. 442.It Ic zs 443Single step 1 instruction (through calls) and show next instruction. 444.El 445.Ss "Examining other processes" 446The following macros access other processes. 447The 448.Nm 449debugger 450does not understand the concept of multiple processes, so they effectively 451bypass the entire 452.Nm 453environment. 454.Bl -tag -width indent 455.It Ic btp Ar pid 456Show a backtrace for the process 457.Ar pid . 458.It Ic btpa 459Show backtraces for all processes in the system. 460.It Ic btpp 461Show a backtrace for the process previously selected with 462.Ic defproc . 463.It Ic btr Ar ebp 464Show a backtrace from the 465.Ar ebp 466address specified. 467.It Ic defproc Ar pid 468Specify the PID of the process for some other commands in this section. 469.It Ic fr Ar frame 470Show frame 471.Ar frame 472of the stack of the process previously selected with 473.Ic defproc . 474.It Ic pcb Ar proc 475Show some PCB contents of the process 476.Ar proc . 477.El 478.Ss "Examining data structures" 479You can use standard 480.Xr gdb 1 481commands to look at most data structures. 482The macros in this section are 483convenience functions which typically display the data in a more readable 484format, or which omit less interesting parts of the structure. 485.Bl -tag -width indent 486.It Ic bp 487Show information about the buffer header pointed to by the variable 488.Va bp 489in the current frame. 490.It Ic bpd 491Show the contents 492.Pq Vt "char *" 493of 494.Va bp->data 495in the current frame. 496.It Ic bpl 497Show detailed information about the buffer header 498.Pq Vt "struct bp" 499pointed at by the local variable 500.Va bp . 501.It Ic bpp Ar bp 502Show summary information about the buffer header 503.Pq Vt "struct bp" 504pointed at by the parameter 505.Ar bp . 506.It Ic bx 507Print a number of fields from the buffer header pointed at in by the pointer 508.Ar bp 509in the current environment. 510.It Ic vdev 511Show some information of the 512.Vt vnode 513pointed to by the local variable 514.Va vp . 515.El 516.Ss "Miscellaneous macros" 517.Bl -tag -width indent 518.It Ic checkmem 519Check unallocated memory for modifications. 520This assumes that the kernel has been compiled with 521.Cd "options DIAGNOSTIC" 522This causes the contents of free memory to be set to 523.Li 0xdeadc0de . 524.It Ic dmesg 525Print the system message buffer. 526This corresponds to the 527.Xr dmesg 8 528utility. 529This macro used to be called 530.Ic msgbuf . 531It can take a very long time over a serial line, 532and it is even slower via firewire 533or local memory due to inefficiencies in 534.Nm . 535When debugging a crash dump or over firewire, it is not necessary to start 536.Nm 537to access the message buffer: instead, use an appropriate variation of 538.Bd -literal -offset indent 539dmesg -M /var/crash/vmcore.0 -N kernel.debug 540dmesg -M /dev/fwmem0.0 -N kernel.debug 541.Ed 542.It Ic kldstat 543Equivalent of the 544.Xr kldstat 8 545utility without options. 546.It Ic pname 547Print the command name of the current process. 548.It Ic ps 549Show process status. 550This corresponds in concept, but not in appearance, to the 551.Xr ps 1 552utility. 553When debugging a crash dump or over firewire, it is not necessary to start 554.Nm 555to display the 556.Xr ps 1 557output: instead, use an appropriate variation of 558.Bd -literal -offset indent 559ps -M /var/crash/vmcore.0 -N kernel.debug 560ps -M /dev/fwmem0.0 -N kernel.debug 561.Ed 562.It Ic y 563Kludge for writing macros. 564When writing macros, it is convenient to paste them 565back into the 566.Nm 567window. 568Unfortunately, if the macro is already defined, 569.Nm 570insists on asking 571.Pp 572.Dl "Redefine foo?" 573.Pp 574It will not give up until you answer 575.Ql y . 576This command is that answer. 577It does nothing else except to print a warning 578message to remind you to remove it again. 579.El 580.Sh AUTHORS 581This man page was written by 582.An "Greg Lehey" Aq grog@FreeBSD.org . 583.Sh SEE ALSO 584.Xr gdb 1 , 585.Xr ps 1 , 586.Xr ddb 4 , 587.Xr firewire 4 , 588.Xr vinumdebug 4 , 589.Xr dconschat 8 , 590.Xr dmesg 8 , 591.Xr fwcontrol 8 , 592.Xr kldload 8 593.Sh BUGS 594The 595.Xr gdb 1 596debugger 597was never designed to debug kernels, and it is not a very good match. 598Many problems exist. 599.Pp 600The 601.Nm 602implementation is very inefficient, and many operations are slow. 603.Pp 604Serial debugging is even slower, and race conditions can make it difficult to 605run the link at more than 9600 bps. 606Firewire connections do not have this problem. 607.Pp 608The debugging macros 609.Dq "just growed" . 610In general, the person who wrote them did so while looking for a specific 611problem, so they may not be general enough, and they may behave badly when used 612in ways for which they were not intended, even if those ways make sense. 613.Pp 614Many of these commands only work on the ia32 architecture. 615