1# Linux Minidump to Core
2
3On Linux, Chromium can use Breakpad to generate minidump files for crashes. It
4is possible to convert the minidump files to core files, and examine the core
5file in gdb, cgdb, or Qtcreator. In the examples below cgdb is assumed but any
6gdb based debugger can be used.
7
8[TOC]
9
10## Creating the core file
11
12Use `minidump-2-core` to convert the minidump file to a core file. On Linux, one
13can build the minidump-2-core target in a Chromium checkout, or alternatively,
14build it in a Google Breakpad checkout.
15
16```shell
17$ ninja -C out/Release minidump-2-core
18$ ./out/Release/minidump-2-core foo.dmp > foo.core
19```
20
21## Retrieving Chrome binaries
22
23If the minidump is from a public build then Googlers can find Google Chrome
24Linux binaries and debugging symbols via https://goto.google.com/chromesymbols.
25Otherwise, use the locally built chrome files. Google Chrome uses the
26_debug link_ method to specify the debugging file. Either way be sure to put
27`chrome` and `chrome.debug` (the stripped debug information) in the same
28directory as the core file so that the debuggers can find them.
29
30For Chrome OS release binaries look for `debug-*.tgz` files on
31GoldenEye.
32
33## Loading the core file into gdb/cgdb
34
35The recommended syntax for loading a core file into gdb/cgdb is as follows,
36specifying both the executable and the core file:
37
38```shell
39$ cgdb chrome foo.core
40```
41
42If the executable is not available then the core file can be loaded on its own
43but debugging options will be limited:
44
45```shell
46$ cgdb -c foo.core
47```
48
49## Loading the core file into Qtcreator
50
51Qtcreator is a full GUI wrapper for gdb and it can also load Chrome's core
52files. From Qtcreator select the Debug menu, Start Debugging, Load Core File...
53and then enter the paths to the core file and executable. Qtcreator has windows
54to display the call stack, locals, registers, etc. For more information on
55debugging with Qtcreator see
56[Getting Started Debugging on Linux.](https://www.youtube.com/watch?v=xTmAknUbpB0)
57
58## Source debugging
59
60If you have a Chromium repo that is synchronized to exactly (or even
61approximately) when the Chrome build was created then you can tell
62`gdb/cgdb/Qtcreator` to load source code. Since all source paths in Chrome are
63relative to the out/Release directory you just need to add that directory to
64your debugger search path, by adding a line similar to this to `~/.gdbinit`:
65
66```
67(gdb) directory /usr/local/chromium/src/out/Release/
68```
69
70## Notes
71
72*   Since the core file is created from a minidump, it is incomplete and the
73    debugger may not know values for variables in memory. Minidump files contain
74    thread stacks so local variables and function parameters should be
75    available, subject to the limitations of optimized builds.
76*   For gdb's `add-symbol-file` command to work, the file must have debugging
77    symbols.
78    *   In case of separate debug files,
79    [the gdb manual](https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html)
80    explains how gdb looks for them.
81*   If the stack trace involve system libraries, the Advanced module loading
82    steps shown below need to be repeated for each library.
83
84## Advanced module loading
85
86If gdb doesn't find shared objects that are needed you can force it to load
87them. In gdb, the `add-symbol-file` command takes a filename and an address. To
88figure out the address, look near the end of `foo.dmp`, which contains a copy of
89`/proc/pid/maps` from the process that crashed.
90
91One quick way to do this is with `grep`. For instance, if the executable is
92`/path/to/chrome`, one can simply run:
93
94```shell
95$ grep -a /path/to/chrome$ foo.dmp
96
977fe749a90000-7fe74d28f000 r-xp 00000000 08:07 289158        /path/to/chrome
987fe74d290000-7fe74d4b7000 r--p 037ff000 08:07 289158        /path/to/chrome
997fe74d4b7000-7fe74d4e0000 rw-p 03a26000 08:07 289158        /path/to/chrome
100```
101
102
103In this case, `7fe749a90000` is the base address for `/path/to/chrome`, but gdb
104takes the start address of the file's text section. To calculate this, one will
105need a copy of `/path/to/chrome`, and run:
106
107```shell
108$ objdump -x /path/to/chrome | grep '\.text' | head -n 1 | tr -s ' ' | \
109    cut -d' ' -f 7
110
111005282c0
112```
113
114
115Now add the two addresses: `7fe749a90000 + 005282c0 = 7fe749fb82c0` and in gdb, run:
116
117```
118(gdb) add-symbol-file /path/to/chrome 0x7fe749fb82c0
119```
120
121Then use gdb as normal.
122
123## Other resources
124
125For more discussion on this process see
126[Debugging a Minidump].
127This page discusses the same process in the context of Chrome OS and many of the
128concepts and techniques overlap.
129
130[Debugging a Minidump](
131https://www.chromium.org/chromium-os/packages/crash-reporting/debugging-a-minidump)
132