1.. highlight:: console
2.. _Troubleshooting:
3
4***************
5Troubleshooting
6***************
7
8First of all, check the logs. Enabling at least the ``warning`` message
9severity may help you to identify some problems. See the :ref:`Logging section`
10for details.
11
12..  _Submitting a bugreport:
13
14Reporting bugs
15==============
16
17If you are unable to solve the problem by yourself, you can submit a
18bugreport to the Knot DNS developers. For security or sensitive issues
19contact the developers directly on
20`knot-dns@labs.nic.cz <mailto:knot-dns@labs.nic.cz>`_.
21All other bugs and questions may be directed to the public Knot DNS users
22mailing list
23(`knot-dns-users@lists.nic.cz <mailto:knot-dns-users@lists.nic.cz>`_) or
24may be entered into the
25`issue tracking system <https://gitlab.nic.cz/knot/knot-dns/issues>`_.
26
27Before anything else, please try to answer the following questions:
28
29* Has it been working?
30* What has changed? System configuration, software updates, network
31  configuration, firewall rules modification, hardware replacement, etc.
32
33The bugreport should contain the answers for the previous questions and in
34addition at least the following information:
35
36* Knot DNS version and type of installation (distribution package, from source,
37  etc.)
38* Operating system, platform, kernel version
39* Relevant basic hardware information (processor, amount of memory, available
40  network devices, etc.)
41* Description of the bug
42* Log output with the highest verbosity (category ``any``, severity ``debug``)
43* Steps to reproduce the bug (if known)
44* Backtrace (if the bug caused a crash or a hang; see the next section)
45
46If possible, please provide a minimal configuration file and zone files which
47can be used to reproduce the bug.
48
49..  _Generating backtrace:
50
51Generating backtrace
52====================
53
54Backtrace carries basic information about the state of the program and how
55the program got where it is. It helps determining the location of the bug in
56the source code.
57
58If you run Knot DNS from distribution packages, make sure the debugging
59symbols for the package are installed. The symbols are usually distributed
60in a separate package.
61
62There are several ways to get the backtrace. One possible way is to extract
63the backtrace from a core dump file. Core dump is a memory snapshot generated
64by the operating system when a process crashes. The generating of core dumps
65must be usually enabled::
66
67    $ ulimit -c unlimited                  # Enable unlimited core dump size
68    $ knotd ...                            # Reproduce the crash
69    ...
70    $ gdb knotd <core-dump-file>           # Start gdb on the core dump
71    (gdb) info threads                     # Get a summary of all threads
72    (gdb) thread apply all bt full         # Extract backtrace from all threads
73    (gdb) quit
74
75To save the backtrace into a file, the following GDB commands can be used::
76
77    (gdb) set pagination off
78    (gdb) set logging file backtrace.txt
79    (gdb) set logging on
80    (gdb) info threads
81    (gdb) thread apply all bt full
82    (gdb) set logging off
83
84To generate a core dump of a running process, the `gcore` utility can be used::
85
86    $ gcore -o <output-file> $(pidof knotd)
87
88Please note that core dumps can be intercepted by an error-collecting system
89service (systemd-coredump, ABRT, Apport, etc.). If you are using such a service,
90consult its documentation about core dump retrieval.
91
92If the error is reproducible, it is also possible to start and inspect the
93server directly in the debugger::
94
95    $ gdb --args knotd -c /etc/knot.conf
96    (gdb) run
97    ...
98
99Alternatively, the debugger can be attached to a running server
100process. This is generally useful when troubleshooting a stuck process::
101
102    $ knotd ...
103    $ gdb --pid $(pidof knotd)
104    (gdb) continue
105    ...
106
107If you fail to get a backtrace of a running process using the previous method,
108you may try the single-purpose ``pstack`` utility::
109
110    $ pstack $(pidof knotd) > backtrace.txt
111