1================================ 2Source Level Debugging with LLVM 3================================ 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11This document is the central repository for all information pertaining to debug 12information in LLVM. It describes the :ref:`actual format that the LLVM debug 13information takes <format>`, which is useful for those interested in creating 14front-ends or dealing directly with the information. Further, this document 15provides specific examples of what debug information for C/C++ looks like. 16 17Philosophy behind LLVM debugging information 18-------------------------------------------- 19 20The idea of the LLVM debugging information is to capture how the important 21pieces of the source-language's Abstract Syntax Tree map onto LLVM code. 22Several design aspects have shaped the solution that appears here. The 23important ones are: 24 25* Debugging information should have very little impact on the rest of the 26 compiler. No transformations, analyses, or code generators should need to 27 be modified because of debugging information. 28 29* LLVM optimizations should interact in :ref:`well-defined and easily described 30 ways <intro_debugopt>` with the debugging information. 31 32* Because LLVM is designed to support arbitrary programming languages, 33 LLVM-to-LLVM tools should not need to know anything about the semantics of 34 the source-level-language. 35 36* Source-level languages are often **widely** different from one another. 37 LLVM should not put any restrictions of the flavor of the source-language, 38 and the debugging information should work with any language. 39 40* With code generator support, it should be possible to use an LLVM compiler 41 to compile a program to native machine code and standard debugging 42 formats. This allows compatibility with traditional machine-code level 43 debuggers, like GDB or DBX. 44 45The approach used by the LLVM implementation is to use a small set of 46:ref:`intrinsic functions <format_common_intrinsics>` to define a mapping 47between LLVM program objects and the source-level objects. The description of 48the source-level program is maintained in LLVM metadata in an 49:ref:`implementation-defined format <ccxx_frontend>` (the C/C++ front-end 50currently uses working draft 7 of the `DWARF 3 standard 51<http://www.eagercon.com/dwarf/dwarf3std.htm>`_). 52 53When a program is being debugged, a debugger interacts with the user and turns 54the stored debug information into source-language specific information. As 55such, a debugger must be aware of the source-language, and is thus tied to a 56specific language or family of languages. 57 58Debug information consumers 59--------------------------- 60 61The role of debug information is to provide meta information normally stripped 62away during the compilation process. This meta information provides an LLVM 63user a relationship between generated code and the original program source 64code. 65 66Currently, there are two backend consumers of debug info: DwarfDebug and 67CodeViewDebug. DwarfDebug produces DWARF suitable for use with GDB, LLDB, and 68other DWARF-based debuggers. :ref:`CodeViewDebug <codeview>` produces CodeView, 69the Microsoft debug info format, which is usable with Microsoft debuggers such 70as Visual Studio and WinDBG. LLVM's debug information format is mostly derived 71from and inspired by DWARF, but it is feasible to translate into other target 72debug info formats such as STABS. 73 74It would also be reasonable to use debug information to feed profiling tools 75for analysis of generated code, or, tools for reconstructing the original 76source from generated code. 77 78.. _intro_debugopt: 79 80Debug information and optimizations 81----------------------------------- 82 83An extremely high priority of LLVM debugging information is to make it interact 84well with optimizations and analysis. In particular, the LLVM debug 85information provides the following guarantees: 86 87* LLVM debug information **always provides information to accurately read 88 the source-level state of the program**, regardless of which LLVM 89 optimizations have been run, and without any modification to the 90 optimizations themselves. However, some optimizations may impact the 91 ability to modify the current state of the program with a debugger, such 92 as setting program variables, or calling functions that have been 93 deleted. 94 95* As desired, LLVM optimizations can be upgraded to be aware of debugging 96 information, allowing them to update the debugging information as they 97 perform aggressive optimizations. This means that, with effort, the LLVM 98 optimizers could optimize debug code just as well as non-debug code. 99 100* LLVM debug information does not prevent optimizations from 101 happening (for example inlining, basic block reordering/merging/cleanup, 102 tail duplication, etc). 103 104* LLVM debug information is automatically optimized along with the rest of 105 the program, using existing facilities. For example, duplicate 106 information is automatically merged by the linker, and unused information 107 is automatically removed. 108 109Basically, the debug information allows you to compile a program with 110"``-O0 -g``" and get full debug information, allowing you to arbitrarily modify 111the program as it executes from a debugger. Compiling a program with 112"``-O3 -g``" gives you full debug information that is always available and 113accurate for reading (e.g., you get accurate stack traces despite tail call 114elimination and inlining), but you might lose the ability to modify the program 115and call functions which were optimized out of the program, or inlined away 116completely. 117 118The :doc:`LLVM test-suite <TestSuiteMakefileGuide>` provides a framework to 119test the optimizer's handling of debugging information. It can be run like 120this: 121 122.. code-block:: bash 123 124 % cd llvm/projects/test-suite/MultiSource/Benchmarks # or some other level 125 % make TEST=dbgopt 126 127This will test impact of debugging information on optimization passes. If 128debugging information influences optimization passes then it will be reported 129as a failure. See :doc:`TestingGuide` for more information on LLVM test 130infrastructure and how to run various tests. 131 132.. _format: 133 134Debugging information format 135============================ 136 137LLVM debugging information has been carefully designed to make it possible for 138the optimizer to optimize the program and debugging information without 139necessarily having to know anything about debugging information. In 140particular, the use of metadata avoids duplicated debugging information from 141the beginning, and the global dead code elimination pass automatically deletes 142debugging information for a function if it decides to delete the function. 143 144To do this, most of the debugging information (descriptors for types, 145variables, functions, source files, etc) is inserted by the language front-end 146in the form of LLVM metadata. 147 148Debug information is designed to be agnostic about the target debugger and 149debugging information representation (e.g. DWARF/Stabs/etc). It uses a generic 150pass to decode the information that represents variables, types, functions, 151namespaces, etc: this allows for arbitrary source-language semantics and 152type-systems to be used, as long as there is a module written for the target 153debugger to interpret the information. 154 155To provide basic functionality, the LLVM debugger does have to make some 156assumptions about the source-level language being debugged, though it keeps 157these to a minimum. The only common features that the LLVM debugger assumes 158exist are `source files <LangRef.html#difile>`_, and `program objects 159<LangRef.html#diglobalvariable>`_. These abstract objects are used by a 160debugger to form stack traces, show information about local variables, etc. 161 162This section of the documentation first describes the representation aspects 163common to any source-language. :ref:`ccxx_frontend` describes the data layout 164conventions used by the C and C++ front-ends. 165 166Debug information descriptors are `specialized metadata nodes 167<LangRef.html#specialized-metadata>`_, first-class subclasses of ``Metadata``. 168 169.. _format_common_intrinsics: 170 171Debugger intrinsic functions 172---------------------------- 173 174LLVM uses several intrinsic functions (name prefixed with "``llvm.dbg``") to 175track source local variables through optimization and code generation. 176 177``llvm.dbg.addr`` 178^^^^^^^^^^^^^^^^^^^^ 179 180.. code-block:: llvm 181 182 void @llvm.dbg.addr(metadata, metadata, metadata) 183 184This intrinsic provides information about a local element (e.g., variable). 185The first argument is metadata holding the address of variable, typically a 186static alloca in the function entry block. The second argument is a 187`local variable <LangRef.html#dilocalvariable>`_ containing a description of 188the variable. The third argument is a `complex expression 189<LangRef.html#diexpression>`_. An `llvm.dbg.addr` intrinsic describes the 190*address* of a source variable. 191 192.. code-block:: text 193 194 %i.addr = alloca i32, align 4 195 call void @llvm.dbg.addr(metadata i32* %i.addr, metadata !1, 196 metadata !DIExpression()), !dbg !2 197 !1 = !DILocalVariable(name: "i", ...) ; int i 198 !2 = !DILocation(...) 199 ... 200 %buffer = alloca [256 x i8], align 8 201 ; The address of i is buffer+64. 202 call void @llvm.dbg.addr(metadata [256 x i8]* %buffer, metadata !3, 203 metadata !DIExpression(DW_OP_plus, 64)), !dbg !4 204 !3 = !DILocalVariable(name: "i", ...) ; int i 205 !4 = !DILocation(...) 206 207A frontend should generate exactly one call to ``llvm.dbg.addr`` at the point 208of declaration of a source variable. Optimization passes that fully promote the 209variable from memory to SSA values will replace this call with possibly 210multiple calls to `llvm.dbg.value`. Passes that delete stores are effectively 211partial promotion, and they will insert a mix of calls to ``llvm.dbg.value`` 212and ``llvm.dbg.addr`` to track the source variable value when it is available. 213After optimization, there may be multiple calls to ``llvm.dbg.addr`` describing 214the program points where the variables lives in memory. All calls for the same 215concrete source variable must agree on the memory location. 216 217 218``llvm.dbg.declare`` 219^^^^^^^^^^^^^^^^^^^^ 220 221.. code-block:: llvm 222 223 void @llvm.dbg.declare(metadata, metadata, metadata) 224 225This intrinsic is identical to `llvm.dbg.addr`, except that there can only be 226one call to `llvm.dbg.declare` for a given concrete `local variable 227<LangRef.html#dilocalvariable>`_. It is not control-dependent, meaning that if 228a call to `llvm.dbg.declare` exists and has a valid location argument, that 229address is considered to be the true home of the variable across its entire 230lifetime. This makes it hard for optimizations to preserve accurate debug info 231in the presence of ``llvm.dbg.declare``, so we are transitioning away from it, 232and we plan to deprecate it in future LLVM releases. 233 234 235``llvm.dbg.value`` 236^^^^^^^^^^^^^^^^^^ 237 238.. code-block:: llvm 239 240 void @llvm.dbg.value(metadata, metadata, metadata) 241 242This intrinsic provides information when a user source variable is set to a new 243value. The first argument is the new value (wrapped as metadata). The second 244argument is a `local variable <LangRef.html#dilocalvariable>`_ containing a 245description of the variable. The third argument is a `complex expression 246<LangRef.html#diexpression>`_. 247 248An `llvm.dbg.value` intrinsic describes the *value* of a source variable 249directly, not its address. Note that the value operand of this intrinsic may 250be indirect (i.e, a pointer to the source variable), provided that interpreting 251the complex expression derives the direct value. 252 253Object lifetimes and scoping 254============================ 255 256In many languages, the local variables in functions can have their lifetimes or 257scopes limited to a subset of a function. In the C family of languages, for 258example, variables are only live (readable and writable) within the source 259block that they are defined in. In functional languages, values are only 260readable after they have been defined. Though this is a very obvious concept, 261it is non-trivial to model in LLVM, because it has no notion of scoping in this 262sense, and does not want to be tied to a language's scoping rules. 263 264In order to handle this, the LLVM debug format uses the metadata attached to 265llvm instructions to encode line number and scoping information. Consider the 266following C fragment, for example: 267 268.. code-block:: c 269 270 1. void foo() { 271 2. int X = 21; 272 3. int Y = 22; 273 4. { 274 5. int Z = 23; 275 6. Z = X; 276 7. } 277 8. X = Y; 278 9. } 279 280.. FIXME: Update the following example to use llvm.dbg.addr once that is the 281 default in clang. 282 283Compiled to LLVM, this function would be represented like this: 284 285.. code-block:: text 286 287 ; Function Attrs: nounwind ssp uwtable 288 define void @foo() #0 !dbg !4 { 289 entry: 290 %X = alloca i32, align 4 291 %Y = alloca i32, align 4 292 %Z = alloca i32, align 4 293 call void @llvm.dbg.declare(metadata i32* %X, metadata !11, metadata !13), !dbg !14 294 store i32 21, i32* %X, align 4, !dbg !14 295 call void @llvm.dbg.declare(metadata i32* %Y, metadata !15, metadata !13), !dbg !16 296 store i32 22, i32* %Y, align 4, !dbg !16 297 call void @llvm.dbg.declare(metadata i32* %Z, metadata !17, metadata !13), !dbg !19 298 store i32 23, i32* %Z, align 4, !dbg !19 299 %0 = load i32, i32* %X, align 4, !dbg !20 300 store i32 %0, i32* %Z, align 4, !dbg !21 301 %1 = load i32, i32* %Y, align 4, !dbg !22 302 store i32 %1, i32* %X, align 4, !dbg !23 303 ret void, !dbg !24 304 } 305 306 ; Function Attrs: nounwind readnone 307 declare void @llvm.dbg.declare(metadata, metadata, metadata) #1 308 309 attributes #0 = { nounwind ssp uwtable "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" } 310 attributes #1 = { nounwind readnone } 311 312 !llvm.dbg.cu = !{!0} 313 !llvm.module.flags = !{!7, !8, !9} 314 !llvm.ident = !{!10} 315 316 !0 = !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, retainedTypes: !2, subprograms: !3, globals: !2, imports: !2) 317 !1 = !DIFile(filename: "/dev/stdin", directory: "/Users/dexonsmith/data/llvm/debug-info") 318 !2 = !{} 319 !3 = !{!4} 320 !4 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !5, isLocal: false, isDefinition: true, scopeLine: 1, isOptimized: false, variables: !2) 321 !5 = !DISubroutineType(types: !6) 322 !6 = !{null} 323 !7 = !{i32 2, !"Dwarf Version", i32 2} 324 !8 = !{i32 2, !"Debug Info Version", i32 3} 325 !9 = !{i32 1, !"PIC Level", i32 2} 326 !10 = !{!"clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)"} 327 !11 = !DILocalVariable(name: "X", scope: !4, file: !1, line: 2, type: !12) 328 !12 = !DIBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed) 329 !13 = !DIExpression() 330 !14 = !DILocation(line: 2, column: 9, scope: !4) 331 !15 = !DILocalVariable(name: "Y", scope: !4, file: !1, line: 3, type: !12) 332 !16 = !DILocation(line: 3, column: 9, scope: !4) 333 !17 = !DILocalVariable(name: "Z", scope: !18, file: !1, line: 5, type: !12) 334 !18 = distinct !DILexicalBlock(scope: !4, file: !1, line: 4, column: 5) 335 !19 = !DILocation(line: 5, column: 11, scope: !18) 336 !20 = !DILocation(line: 6, column: 11, scope: !18) 337 !21 = !DILocation(line: 6, column: 9, scope: !18) 338 !22 = !DILocation(line: 8, column: 9, scope: !4) 339 !23 = !DILocation(line: 8, column: 7, scope: !4) 340 !24 = !DILocation(line: 9, column: 3, scope: !4) 341 342 343This example illustrates a few important details about LLVM debugging 344information. In particular, it shows how the ``llvm.dbg.declare`` intrinsic and 345location information, which are attached to an instruction, are applied 346together to allow a debugger to analyze the relationship between statements, 347variable definitions, and the code used to implement the function. 348 349.. code-block:: llvm 350 351 call void @llvm.dbg.declare(metadata i32* %X, metadata !11, metadata !13), !dbg !14 352 ; [debug line = 2:7] [debug variable = X] 353 354The first intrinsic ``%llvm.dbg.declare`` encodes debugging information for the 355variable ``X``. The metadata ``!dbg !14`` attached to the intrinsic provides 356scope information for the variable ``X``. 357 358.. code-block:: text 359 360 !14 = !DILocation(line: 2, column: 9, scope: !4) 361 !4 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !5, 362 isLocal: false, isDefinition: true, scopeLine: 1, 363 isOptimized: false, variables: !2) 364 365Here ``!14`` is metadata providing `location information 366<LangRef.html#dilocation>`_. In this example, scope is encoded by ``!4``, a 367`subprogram descriptor <LangRef.html#disubprogram>`_. This way the location 368information attached to the intrinsics indicates that the variable ``X`` is 369declared at line number 2 at a function level scope in function ``foo``. 370 371Now lets take another example. 372 373.. code-block:: llvm 374 375 call void @llvm.dbg.declare(metadata i32* %Z, metadata !17, metadata !13), !dbg !19 376 ; [debug line = 5:9] [debug variable = Z] 377 378The third intrinsic ``%llvm.dbg.declare`` encodes debugging information for 379variable ``Z``. The metadata ``!dbg !19`` attached to the intrinsic provides 380scope information for the variable ``Z``. 381 382.. code-block:: text 383 384 !18 = distinct !DILexicalBlock(scope: !4, file: !1, line: 4, column: 5) 385 !19 = !DILocation(line: 5, column: 11, scope: !18) 386 387Here ``!19`` indicates that ``Z`` is declared at line number 5 and column 388number 11 inside of lexical scope ``!18``. The lexical scope itself resides 389inside of subprogram ``!4`` described above. 390 391The scope information attached with each instruction provides a straightforward 392way to find instructions covered by a scope. 393 394Object lifetime in optimized code 395================================= 396 397In the example above, every variable assignment uniquely corresponds to a 398memory store to the variable's position on the stack. However in heavily 399optimized code LLVM promotes most variables into SSA values, which can 400eventually be placed in physical registers or memory locations. To track SSA 401values through compilation, when objects are promoted to SSA values an 402``llvm.dbg.value`` intrinsic is created for each assignment, recording the 403variable's new location. Compared with the ``llvm.dbg.declare`` intrinsic: 404 405* A dbg.value terminates the effect of any preceeding dbg.values for (any 406 overlapping fragments of) the specified variable. 407* The dbg.value's position in the IR defines where in the instruction stream 408 the variable's value changes. 409* Operands can be constants, indicating the variable is assigned a 410 constant value. 411 412Care must be taken to update ``llvm.dbg.value`` intrinsics when optimization 413passes alter or move instructions and blocks -- the developer could observe such 414changes reflected in the value of variables when debugging the program. For any 415execution of the optimized program, the set of variable values presented to the 416developer by the debugger should not show a state that would never have existed 417in the execution of the unoptimized program, given the same input. Doing so 418risks misleading the developer by reporting a state that does not exist, 419damaging their understanding of the optimized program and undermining their 420trust in the debugger. 421 422Sometimes perfectly preserving variable locations is not possible, often when a 423redundant calculation is optimized out. In such cases, a ``llvm.dbg.value`` 424with operand ``undef`` should be used, to terminate earlier variable locations 425and let the debugger present ``optimized out`` to the developer. Withholding 426these potentially stale variable values from the developer diminishes the 427amount of available debug information, but increases the reliability of the 428remaining information. 429 430To illustrate some potential issues, consider the following example: 431 432.. code-block:: llvm 433 434 define i32 @foo(i32 %bar, i1 %cond) { 435 entry: 436 call @llvm.dbg.value(metadata i32 0, metadata !1, metadata !2) 437 br i1 %cond, label %truebr, label %falsebr 438 truebr: 439 %tval = add i32 %bar, 1 440 call @llvm.dbg.value(metadata i32 %tval, metadata !1, metadata !2) 441 %g1 = call i32 @gazonk() 442 br label %exit 443 falsebr: 444 %fval = add i32 %bar, 2 445 call @llvm.dbg.value(metadata i32 %fval, metadata !1, metadata !2) 446 %g2 = call i32 @gazonk() 447 br label %exit 448 exit: 449 %merge = phi [ %tval, %truebr ], [ %fval, %falsebr ] 450 %g = phi [ %g1, %truebr ], [ %g2, %falsebr ] 451 call @llvm.dbg.value(metadata i32 %merge, metadata !1, metadata !2) 452 call @llvm.dbg.value(metadata i32 %g, metadata !3, metadata !2) 453 %plusten = add i32 %merge, 10 454 %toret = add i32 %plusten, %g 455 call @llvm.dbg.value(metadata i32 %toret, metadata !1, metadata !2) 456 ret i32 %toret 457 } 458 459Containing two source-level variables in ``!1`` and ``!3``. The function could, 460perhaps, be optimized into the following code: 461 462.. code-block:: llvm 463 464 define i32 @foo(i32 %bar, i1 %cond) { 465 entry: 466 %g = call i32 @gazonk() 467 %addoper = select i1 %cond, i32 11, i32 12 468 %plusten = add i32 %bar, %addoper 469 %toret = add i32 %plusten, %g 470 ret i32 %toret 471 } 472 473What ``llvm.dbg.value`` intrinsics should be placed to represent the original variable 474locations in this code? Unfortunately the the second, third and fourth 475dbg.values for ``!1`` in the source function have had their operands 476(%tval, %fval, %merge) optimized out. Assuming we cannot recover them, we 477might consider this placement of dbg.values: 478 479.. code-block:: llvm 480 481 define i32 @foo(i32 %bar, i1 %cond) { 482 entry: 483 call @llvm.dbg.value(metadata i32 0, metadata !1, metadata !2) 484 %g = call i32 @gazonk() 485 call @llvm.dbg.value(metadata i32 %g, metadata !3, metadata !2) 486 %addoper = select i1 %cond, i32 11, i32 12 487 %plusten = add i32 %bar, %addoper 488 %toret = add i32 %plusten, %g 489 call @llvm.dbg.value(metadata i32 %toret, metadata !1, metadata !2) 490 ret i32 %toret 491 } 492 493However, this will cause ``!3`` to have the return value of ``@gazonk()`` at 494the same time as ``!1`` has the constant value zero -- a pair of assignments 495that never occurred in the unoptimized program. To avoid this, we must terminate 496the range that ``!1`` has the constant value assignment by inserting an undef 497dbg.value before the dbg.value for ``!3``: 498 499.. code-block:: llvm 500 501 define i32 @foo(i32 %bar, i1 %cond) { 502 entry: 503 call @llvm.dbg.value(metadata i32 0, metadata !1, metadata !2) 504 %g = call i32 @gazonk() 505 call @llvm.dbg.value(metadata i32 undef, metadata !1, metadata !2) 506 call @llvm.dbg.value(metadata i32 %g, metadata !3, metadata !2) 507 %addoper = select i1 %cond, i32 11, i32 12 508 %plusten = add i32 %bar, %addoper 509 %toret = add i32 %plusten, %g 510 call @llvm.dbg.value(metadata i32 %toret, metadata !1, metadata !2) 511 ret i32 %toret 512 } 513 514In general, if any dbg.value has its operand optimized out and cannot be 515recovered, then an undef dbg.value is necessary to terminate earlier variable 516locations. Additional undef dbg.values may be necessary when the debugger can 517observe re-ordering of assignments. 518 519How variable location metadata is transformed during CodeGen 520============================================================ 521 522LLVM preserves debug information throughout mid-level and backend passes, 523ultimately producing a mapping between source-level information and 524instruction ranges. This 525is relatively straightforwards for line number information, as mapping 526instructions to line numbers is a simple association. For variable locations 527however the story is more complex. As each ``llvm.dbg.value`` intrinsic 528represents a source-level assignment of a value to a source variable, the 529variable location intrinsics effectively embed a small imperative program 530within the LLVM IR. By the end of CodeGen, this becomes a mapping from each 531variable to their machine locations over ranges of instructions. 532From IR to object emission, the major transformations which affect variable 533location fidelity are: 534 5351. Instruction Selection 5362. Register allocation 5373. Block layout 538 539each of which are discussed below. In addition, instruction scheduling can 540significantly change the ordering of the program, and occurs in a number of 541different passes. 542 543Some variable locations are not transformed during CodeGen. Stack locations 544specified by ``llvm.dbg.declare`` are valid and unchanging for the entire 545duration of the function, and are recorded in a simple MachineFunction table. 546Location changes in the prologue and epilogue of a function are also ignored: 547frame setup and destruction may take several instructions, require a 548disproportionate amount of debugging information in the output binary to 549describe, and should be stepped over by debuggers anyway. 550 551Variable locations in Instruction Selection and MIR 552--------------------------------------------------- 553 554Instruction selection creates a MIR function from an IR function, and just as 555it transforms ``intermediate`` instructions into machine instructions, so must 556``intermediate`` variable locations become machine variable locations. 557Within IR, variable locations are always identified by a Value, but in MIR 558there can be different types of variable locations. In addition, some IR 559locations become unavailable, for example if the operation of multiple IR 560instructions are combined into one machine instruction (such as 561multiply-and-accumulate) then intermediate Values are lost. To track variable 562locations through instruction selection, they are first separated into 563locations that do not depend on code generation (constants, stack locations, 564allocated virtual registers) and those that do. For those that do, debug 565metadata is attached to SDNodes in SelectionDAGs. After instruction selection 566has occurred and a MIR function is created, if the SDNode associated with debug 567metadata is allocated a virtual register, that virtual register is used as the 568variable location. If the SDNode is folded into a machine instruction or 569otherwise transformed into a non-register, the variable location becomes 570unavailable. 571 572Locations that are unavailable are treated as if they have been optimized out: 573in IR the location would be assigned ``undef`` by a debug intrinsic, and in MIR 574the equivalent location is used. 575 576After MIR locations are assigned to each variable, machine pseudo-instructions 577corresponding to each ``llvm.dbg.value`` and ``llvm.dbg.addr`` intrinsic are 578inserted. These ``DBG_VALUE`` instructions appear thus: 579 580.. code-block:: text 581 582 DBG_VALUE %1, $noreg, !123, !DIExpression() 583 584And have the following operands: 585 * The first operand can record the variable location as a register, 586 a frame index, an immediate, or the base address register if the original 587 debug intrinsic referred to memory. ``$noreg`` indicates the variable 588 location is undefined, equivalent to an ``undef`` dbg.value operand. 589 * The type of the second operand indicates whether the variable location is 590 directly referred to by the DBG_VALUE, or whether it is indirect. The 591 ``$noreg`` register signifies the former, an immediate operand (0) the 592 latter. 593 * Operand 3 is the Variable field of the original debug intrinsic. 594 * Operand 4 is the Expression field of the original debug intrinsic. 595 596The position at which the DBG_VALUEs are inserted should correspond to the 597positions of their matching ``llvm.dbg.value`` intrinsics in the IR block. As 598with optimization, LLVM aims to preserve the order in which variable 599assignments occurred in the source program. However SelectionDAG performs some 600instruction scheduling, which can reorder assignments (discussed below). 601Function parameter locations are moved to the beginning of the function if 602they're not already, to ensure they're immediately available on function entry. 603 604To demonstrate variable locations during instruction selection, consider 605the following example: 606 607.. code-block:: llvm 608 609 define i32 @foo(i32* %addr) { 610 entry: 611 call void @llvm.dbg.value(metadata i32 0, metadata !3, metadata !DIExpression()), !dbg !5 612 br label %bb1, !dbg !5 613 614 bb1: ; preds = %bb1, %entry 615 %bar.0 = phi i32 [ 0, %entry ], [ %add, %bb1 ] 616 call void @llvm.dbg.value(metadata i32 %bar.0, metadata !3, metadata !DIExpression()), !dbg !5 617 %addr1 = getelementptr i32, i32 *%addr, i32 1, !dbg !5 618 call void @llvm.dbg.value(metadata i32 *%addr1, metadata !3, metadata !DIExpression()), !dbg !5 619 %loaded1 = load i32, i32* %addr1, !dbg !5 620 %addr2 = getelementptr i32, i32 *%addr, i32 %bar.0, !dbg !5 621 call void @llvm.dbg.value(metadata i32 *%addr2, metadata !3, metadata !DIExpression()), !dbg !5 622 %loaded2 = load i32, i32* %addr2, !dbg !5 623 %add = add i32 %bar.0, 1, !dbg !5 624 call void @llvm.dbg.value(metadata i32 %add, metadata !3, metadata !DIExpression()), !dbg !5 625 %added = add i32 %loaded1, %loaded2 626 %cond = icmp ult i32 %added, %bar.0, !dbg !5 627 br i1 %cond, label %bb1, label %bb2, !dbg !5 628 629 bb2: ; preds = %bb1 630 ret i32 0, !dbg !5 631 } 632 633If one compiles this IR with ``llc -o - -start-after=codegen-prepare -stop-after=expand-isel-pseudos -mtriple=x86_64--``, the following MIR is produced: 634 635.. code-block:: text 636 637 bb.0.entry: 638 successors: %bb.1(0x80000000) 639 liveins: $rdi 640 641 %2:gr64 = COPY $rdi 642 %3:gr32 = MOV32r0 implicit-def dead $eflags 643 DBG_VALUE 0, $noreg, !3, !DIExpression(), debug-location !5 644 645 bb.1.bb1: 646 successors: %bb.1(0x7c000000), %bb.2(0x04000000) 647 648 %0:gr32 = PHI %3, %bb.0, %1, %bb.1 649 DBG_VALUE %0, $noreg, !3, !DIExpression(), debug-location !5 650 DBG_VALUE %2, $noreg, !3, !DIExpression(DW_OP_plus_uconst, 4, DW_OP_stack_value), debug-location !5 651 %4:gr32 = MOV32rm %2, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 652 %5:gr64_nosp = MOVSX64rr32 %0, debug-location !5 653 DBG_VALUE $noreg, $noreg, !3, !DIExpression(), debug-location !5 654 %1:gr32 = INC32r %0, implicit-def dead $eflags, debug-location !5 655 DBG_VALUE %1, $noreg, !3, !DIExpression(), debug-location !5 656 %6:gr32 = ADD32rm %4, %2, 4, killed %5, 0, $noreg, implicit-def dead $eflags :: (load 4 from %ir.addr2) 657 %7:gr32 = SUB32rr %6, %0, implicit-def $eflags, debug-location !5 658 JB_1 %bb.1, implicit $eflags, debug-location !5 659 JMP_1 %bb.2, debug-location !5 660 661 bb.2.bb2: 662 %8:gr32 = MOV32r0 implicit-def dead $eflags 663 $eax = COPY %8, debug-location !5 664 RET 0, $eax, debug-location !5 665 666Observe first that there is a DBG_VALUE instruction for every ``llvm.dbg.value`` 667intrinsic in the source IR, ensuring no source level assignments go missing. 668Then consider the different ways in which variable locations have been recorded: 669 670* For the first dbg.value an immediate operand is used to record a zero value. 671* The dbg.value of the PHI instruction leads to a DBG_VALUE of virtual register 672 ``%0``. 673* The first GEP has its effect folded into the first load instruction 674 (as a 4-byte offset), but the variable location is salvaged by folding 675 the GEPs effect into the DIExpression. 676* The second GEP is also folded into the corresponding load. However, it is 677 insufficiently simple to be salvaged, and is emitted as a ``$noreg`` 678 DBG_VALUE, indicating that the variable takes on an undefined location. 679* The final dbg.value has its Value placed in virtual register ``%1``. 680 681Instruction Scheduling 682---------------------- 683 684A number of passes can reschedule instructions, notably instruction selection 685and the pre-and-post RA machine schedulers. Instruction scheduling can 686significantly change the nature of the program -- in the (very unlikely) worst 687case the instruction sequence could be completely reversed. In such 688circumstances LLVM follows the principle applied to optimizations, that it is 689better for the debugger not to display any state than a misleading state. 690Thus, whenever instructions are advanced in order of execution, any 691corresponding DBG_VALUE is kept in its original position, and if an instruction 692is delayed then the variable is given an undefined location for the duration 693of the delay. To illustrate, consider this pseudo-MIR: 694 695.. code-block:: text 696 697 %1:gr32 = MOV32rm %0, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 698 DBG_VALUE %1, $noreg, !1, !2 699 %4:gr32 = ADD32rr %3, %2, implicit-def dead $eflags 700 DBG_VALUE %4, $noreg, !3, !4 701 %7:gr32 = SUB32rr %6, %5, implicit-def dead $eflags 702 DBG_VALUE %7, $noreg, !5, !6 703 704Imagine that the SUB32rr were moved forward to give us the following MIR: 705 706.. code-block:: text 707 708 %7:gr32 = SUB32rr %6, %5, implicit-def dead $eflags 709 %1:gr32 = MOV32rm %0, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 710 DBG_VALUE %1, $noreg, !1, !2 711 %4:gr32 = ADD32rr %3, %2, implicit-def dead $eflags 712 DBG_VALUE %4, $noreg, !3, !4 713 DBG_VALUE %7, $noreg, !5, !6 714 715In this circumstance LLVM would leave the MIR as shown above. Were we to move 716the DBG_VALUE of virtual register %7 upwards with the SUB32rr, we would re-order 717assignments and introduce a new state of the program. Wheras with the solution 718above, the debugger will see one fewer combination of variable values, because 719``!3`` and ``!5`` will change value at the same time. This is preferred over 720misrepresenting the original program. 721 722In comparison, if one sunk the MOV32rm, LLVM would produce the following: 723 724.. code-block:: text 725 726 DBG_VALUE $noreg, $noreg, !1, !2 727 %4:gr32 = ADD32rr %3, %2, implicit-def dead $eflags 728 DBG_VALUE %4, $noreg, !3, !4 729 %7:gr32 = SUB32rr %6, %5, implicit-def dead $eflags 730 DBG_VALUE %7, $noreg, !5, !6 731 %1:gr32 = MOV32rm %0, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 732 DBG_VALUE %1, $noreg, !1, !2 733 734Here, to avoid presenting a state in which the first assignment to ``!1`` 735disappears, the DBG_VALUE at the top of the block assigns the variable the 736undefined location, until its value is available at the end of the block where 737an additional DBG_VALUE is added. Were any other DBG_VALUE for ``!1`` to occur 738in the instructions that the MOV32rm was sunk past, the DBG_VALUE for ``%1`` 739would be dropped and the debugger would never observe it in the variable. This 740accurately reflects that the value is not available during the corresponding 741portion of the original program. 742 743Variable locations during Register Allocation 744--------------------------------------------- 745 746To avoid debug instructions interfering with the register allocator, the 747LiveDebugVariables pass extracts variable locations from a MIR function and 748deletes the corresponding DBG_VALUE instructions. Some localized copy 749propagation is performed within blocks. After register allocation, the 750VirtRegRewriter pass re-inserts DBG_VALUE instructions in their orignal 751positions, translating virtual register references into their physical 752machine locations. To avoid encoding incorrect variable locations, in this 753pass any DBG_VALUE of a virtual register that is not live, is replaced by 754the undefined location. 755 756LiveDebugValues expansion of variable locations 757----------------------------------------------- 758 759After all optimizations have run and shortly before emission, the 760LiveDebugValues pass runs to achieve two aims: 761 762* To propagate the location of variables through copies and register spills, 763* For every block, to record every valid variable location in that block. 764 765After this pass the DBG_VALUE instruction changes meaning: rather than 766corresponding to a source-level assignment where the variable may change value, 767it asserts the location of a variable in a block, and loses effect outside the 768block. Propagating variable locations through copies and spills is 769straightforwards: determining the variable location in every basic block 770requries the consideraton of control flow. Consider the following IR, which 771presents several difficulties: 772 773.. code-block:: text 774 775 define dso_local i32 @foo(i1 %cond, i32 %input) !dbg !12 { 776 entry: 777 br i1 %cond, label %truebr, label %falsebr 778 779 bb1: 780 %value = phi i32 [ %value1, %truebr ], [ %value2, %falsebr ] 781 br label %exit, !dbg !26 782 783 truebr: 784 call void @llvm.dbg.value(metadata i32 %input, metadata !30, metadata !DIExpression()), !dbg !24 785 call void @llvm.dbg.value(metadata i32 1, metadata !23, metadata !DIExpression()), !dbg !24 786 %value1 = add i32 %input, 1 787 br label %bb1 788 789 falsebr: 790 call void @llvm.dbg.value(metadata i32 %input, metadata !30, metadata !DIExpression()), !dbg !24 791 call void @llvm.dbg.value(metadata i32 2, metadata !23, metadata !DIExpression()), !dbg !24 792 %value = add i32 %input, 2 793 br label %bb1 794 795 exit: 796 ret i32 %value, !dbg !30 797 } 798 799Here the difficulties are: 800 801* The control flow is roughly the opposite of basic block order 802* The value of the ``!23`` variable merges into ``%bb1``, but there is no PHI 803 node 804 805As mentioned above, the ``llvm.dbg.value`` intrinsics essentially form an 806imperative program embedded in the IR, with each intrinsic defining a variable 807location. This *could* be converted to an SSA form by mem2reg, in the same way 808that it uses use-def chains to identify control flow merges and insert phi 809nodes for IR Values. However, because debug variable locations are defined for 810every machine instruction, in effect every IR instruction uses every variable 811location, which would lead to a large number of debugging intrinsics being 812generated. 813 814Examining the example above, variable ``!30`` is assigned ``%input`` on both 815conditional paths through the function, while ``!23`` is assigned differing 816constant values on either path. Where control flow merges in ``%bb1`` we would 817want ``!30`` to keep its location (``%input``), but ``!23`` to become undefined 818as we cannot determine at runtime what value it should have in %bb1 without 819inserting a PHI node. mem2reg does not insert the PHI node to avoid changing 820codegen when debugging is enabled, and does not insert the other dbg.values 821to avoid adding very large numbers of intrinsics. 822 823Instead, LiveDebugValues determines variable locations when control 824flow merges. A dataflow analysis is used to propagate locations between blocks: 825when control flow merges, if a variable has the same location in all 826predecessors then that location is propagated into the successor. If the 827predecessor locations disagree, the location becomes undefined. 828 829Once LiveDebugValues has run, every block should have all valid variable 830locations described by DBG_VALUE instructions within the block. Very little 831effort is then required by supporting classes (such as 832DbgEntityHistoryCalculator) to build a map of each instruction to every 833valid variable location, without the need to consider control flow. From 834the example above, it is otherwise difficult to determine that the location 835of variable ``!30`` should flow "up" into block ``%bb1``, but that the location 836of variable ``!23`` should not flow "down" into the ``%exit`` block. 837 838.. _ccxx_frontend: 839 840C/C++ front-end specific debug information 841========================================== 842 843The C and C++ front-ends represent information about the program in a format 844that is effectively identical to `DWARF 3.0 845<http://www.eagercon.com/dwarf/dwarf3std.htm>`_ in terms of information 846content. This allows code generators to trivially support native debuggers by 847generating standard dwarf information, and contains enough information for 848non-dwarf targets to translate it as needed. 849 850This section describes the forms used to represent C and C++ programs. Other 851languages could pattern themselves after this (which itself is tuned to 852representing programs in the same way that DWARF 3 does), or they could choose 853to provide completely different forms if they don't fit into the DWARF model. 854As support for debugging information gets added to the various LLVM 855source-language front-ends, the information used should be documented here. 856 857The following sections provide examples of a few C/C++ constructs and the debug 858information that would best describe those constructs. The canonical 859references are the ``DIDescriptor`` classes defined in 860``include/llvm/IR/DebugInfo.h`` and the implementations of the helper functions 861in ``lib/IR/DIBuilder.cpp``. 862 863C/C++ source file information 864----------------------------- 865 866``llvm::Instruction`` provides easy access to metadata attached with an 867instruction. One can extract line number information encoded in LLVM IR using 868``Instruction::getDebugLoc()`` and ``DILocation::getLine()``. 869 870.. code-block:: c++ 871 872 if (DILocation *Loc = I->getDebugLoc()) { // Here I is an LLVM instruction 873 unsigned Line = Loc->getLine(); 874 StringRef File = Loc->getFilename(); 875 StringRef Dir = Loc->getDirectory(); 876 bool ImplicitCode = Loc->isImplicitCode(); 877 } 878 879When the flag ImplicitCode is true then it means that the Instruction has been 880added by the front-end but doesn't correspond to source code written by the user. For example 881 882.. code-block:: c++ 883 884 if (MyBoolean) { 885 MyObject MO; 886 ... 887 } 888 889At the end of the scope the MyObject's destructor is called but it isn't written 890explicitly. This information is useful to avoid to have counters on brackets when 891making code coverage. 892 893C/C++ global variable information 894--------------------------------- 895 896Given an integer global variable declared as follows: 897 898.. code-block:: c 899 900 _Alignas(8) int MyGlobal = 100; 901 902a C/C++ front-end would generate the following descriptors: 903 904.. code-block:: text 905 906 ;; 907 ;; Define the global itself. 908 ;; 909 @MyGlobal = global i32 100, align 8, !dbg !0 910 911 ;; 912 ;; List of debug info of globals 913 ;; 914 !llvm.dbg.cu = !{!1} 915 916 ;; Some unrelated metadata. 917 !llvm.module.flags = !{!6, !7} 918 !llvm.ident = !{!8} 919 920 ;; Define the global variable itself 921 !0 = distinct !DIGlobalVariable(name: "MyGlobal", scope: !1, file: !2, line: 1, type: !5, isLocal: false, isDefinition: true, align: 64) 922 923 ;; Define the compile unit. 924 !1 = distinct !DICompileUnit(language: DW_LANG_C99, file: !2, 925 producer: "clang version 4.0.0", 926 isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, 927 enums: !3, globals: !4) 928 929 ;; 930 ;; Define the file 931 ;; 932 !2 = !DIFile(filename: "/dev/stdin", 933 directory: "/Users/dexonsmith/data/llvm/debug-info") 934 935 ;; An empty array. 936 !3 = !{} 937 938 ;; The Array of Global Variables 939 !4 = !{!0} 940 941 ;; 942 ;; Define the type 943 ;; 944 !5 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed) 945 946 ;; Dwarf version to output. 947 !6 = !{i32 2, !"Dwarf Version", i32 4} 948 949 ;; Debug info schema version. 950 !7 = !{i32 2, !"Debug Info Version", i32 3} 951 952 ;; Compiler identification 953 !8 = !{!"clang version 4.0.0"} 954 955 956The align value in DIGlobalVariable description specifies variable alignment in 957case it was forced by C11 _Alignas(), C++11 alignas() keywords or compiler 958attribute __attribute__((aligned ())). In other case (when this field is missing) 959alignment is considered default. This is used when producing DWARF output 960for DW_AT_alignment value. 961 962C/C++ function information 963-------------------------- 964 965Given a function declared as follows: 966 967.. code-block:: c 968 969 int main(int argc, char *argv[]) { 970 return 0; 971 } 972 973a C/C++ front-end would generate the following descriptors: 974 975.. code-block:: text 976 977 ;; 978 ;; Define the anchor for subprograms. 979 ;; 980 !4 = !DISubprogram(name: "main", scope: !1, file: !1, line: 1, type: !5, 981 isLocal: false, isDefinition: true, scopeLine: 1, 982 flags: DIFlagPrototyped, isOptimized: false, 983 variables: !2) 984 985 ;; 986 ;; Define the subprogram itself. 987 ;; 988 define i32 @main(i32 %argc, i8** %argv) !dbg !4 { 989 ... 990 } 991 992Fortran specific debug information 993================================== 994 995Fortran function information 996---------------------------- 997 998There are a few DWARF attributes defined to support client debugging of Fortran programs. LLVM can generate (or omit) the appropriate DWARF attributes for the prefix-specs of ELEMENTAL, PURE, IMPURE, RECURSIVE, and NON_RECURSIVE. This is done by using the spFlags values: DISPFlagElemental, DISPFlagPure, and DISPFlagRecursive. 999 1000.. code-block:: fortran 1001 1002 elemental function elem_func(a) 1003 1004a Fortran front-end would generate the following descriptors: 1005 1006.. code-block:: text 1007 1008 !11 = distinct !DISubprogram(name: "subroutine2", scope: !1, file: !1, 1009 line: 5, type: !8, scopeLine: 6, 1010 spFlags: DISPFlagDefinition | DISPFlagElemental, unit: !0, 1011 retainedNodes: !2) 1012 1013and this will materialize an additional DWARF attribute as: 1014 1015.. code-block:: text 1016 1017 DW_TAG_subprogram [3] 1018 DW_AT_low_pc [DW_FORM_addr] (0x0000000000000010 ".text") 1019 DW_AT_high_pc [DW_FORM_data4] (0x00000001) 1020 ... 1021 DW_AT_elemental [DW_FORM_flag_present] (true) 1022 1023Debugging information format 1024============================ 1025 1026Debugging Information Extension for Objective C Properties 1027---------------------------------------------------------- 1028 1029Introduction 1030^^^^^^^^^^^^ 1031 1032Objective C provides a simpler way to declare and define accessor methods using 1033declared properties. The language provides features to declare a property and 1034to let compiler synthesize accessor methods. 1035 1036The debugger lets developer inspect Objective C interfaces and their instance 1037variables and class variables. However, the debugger does not know anything 1038about the properties defined in Objective C interfaces. The debugger consumes 1039information generated by compiler in DWARF format. The format does not support 1040encoding of Objective C properties. This proposal describes DWARF extensions to 1041encode Objective C properties, which the debugger can use to let developers 1042inspect Objective C properties. 1043 1044Proposal 1045^^^^^^^^ 1046 1047Objective C properties exist separately from class members. A property can be 1048defined only by "setter" and "getter" selectors, and be calculated anew on each 1049access. Or a property can just be a direct access to some declared ivar. 1050Finally it can have an ivar "automatically synthesized" for it by the compiler, 1051in which case the property can be referred to in user code directly using the 1052standard C dereference syntax as well as through the property "dot" syntax, but 1053there is no entry in the ``@interface`` declaration corresponding to this ivar. 1054 1055To facilitate debugging, these properties we will add a new DWARF TAG into the 1056``DW_TAG_structure_type`` definition for the class to hold the description of a 1057given property, and a set of DWARF attributes that provide said description. 1058The property tag will also contain the name and declared type of the property. 1059 1060If there is a related ivar, there will also be a DWARF property attribute placed 1061in the ``DW_TAG_member`` DIE for that ivar referring back to the property TAG 1062for that property. And in the case where the compiler synthesizes the ivar 1063directly, the compiler is expected to generate a ``DW_TAG_member`` for that 1064ivar (with the ``DW_AT_artificial`` set to 1), whose name will be the name used 1065to access this ivar directly in code, and with the property attribute pointing 1066back to the property it is backing. 1067 1068The following examples will serve as illustration for our discussion: 1069 1070.. code-block:: objc 1071 1072 @interface I1 { 1073 int n2; 1074 } 1075 1076 @property int p1; 1077 @property int p2; 1078 @end 1079 1080 @implementation I1 1081 @synthesize p1; 1082 @synthesize p2 = n2; 1083 @end 1084 1085This produces the following DWARF (this is a "pseudo dwarfdump" output): 1086 1087.. code-block:: none 1088 1089 0x00000100: TAG_structure_type [7] * 1090 AT_APPLE_runtime_class( 0x10 ) 1091 AT_name( "I1" ) 1092 AT_decl_file( "Objc_Property.m" ) 1093 AT_decl_line( 3 ) 1094 1095 0x00000110 TAG_APPLE_property 1096 AT_name ( "p1" ) 1097 AT_type ( {0x00000150} ( int ) ) 1098 1099 0x00000120: TAG_APPLE_property 1100 AT_name ( "p2" ) 1101 AT_type ( {0x00000150} ( int ) ) 1102 1103 0x00000130: TAG_member [8] 1104 AT_name( "_p1" ) 1105 AT_APPLE_property ( {0x00000110} "p1" ) 1106 AT_type( {0x00000150} ( int ) ) 1107 AT_artificial ( 0x1 ) 1108 1109 0x00000140: TAG_member [8] 1110 AT_name( "n2" ) 1111 AT_APPLE_property ( {0x00000120} "p2" ) 1112 AT_type( {0x00000150} ( int ) ) 1113 1114 0x00000150: AT_type( ( int ) ) 1115 1116Note, the current convention is that the name of the ivar for an 1117auto-synthesized property is the name of the property from which it derives 1118with an underscore prepended, as is shown in the example. But we actually 1119don't need to know this convention, since we are given the name of the ivar 1120directly. 1121 1122Also, it is common practice in ObjC to have different property declarations in 1123the @interface and @implementation - e.g. to provide a read-only property in 1124the interface,and a read-write interface in the implementation. In that case, 1125the compiler should emit whichever property declaration will be in force in the 1126current translation unit. 1127 1128Developers can decorate a property with attributes which are encoded using 1129``DW_AT_APPLE_property_attribute``. 1130 1131.. code-block:: objc 1132 1133 @property (readonly, nonatomic) int pr; 1134 1135.. code-block:: none 1136 1137 TAG_APPLE_property [8] 1138 AT_name( "pr" ) 1139 AT_type ( {0x00000147} (int) ) 1140 AT_APPLE_property_attribute (DW_APPLE_PROPERTY_readonly, DW_APPLE_PROPERTY_nonatomic) 1141 1142The setter and getter method names are attached to the property using 1143``DW_AT_APPLE_property_setter`` and ``DW_AT_APPLE_property_getter`` attributes. 1144 1145.. code-block:: objc 1146 1147 @interface I1 1148 @property (setter=myOwnP3Setter:) int p3; 1149 -(void)myOwnP3Setter:(int)a; 1150 @end 1151 1152 @implementation I1 1153 @synthesize p3; 1154 -(void)myOwnP3Setter:(int)a{ } 1155 @end 1156 1157The DWARF for this would be: 1158 1159.. code-block:: none 1160 1161 0x000003bd: TAG_structure_type [7] * 1162 AT_APPLE_runtime_class( 0x10 ) 1163 AT_name( "I1" ) 1164 AT_decl_file( "Objc_Property.m" ) 1165 AT_decl_line( 3 ) 1166 1167 0x000003cd TAG_APPLE_property 1168 AT_name ( "p3" ) 1169 AT_APPLE_property_setter ( "myOwnP3Setter:" ) 1170 AT_type( {0x00000147} ( int ) ) 1171 1172 0x000003f3: TAG_member [8] 1173 AT_name( "_p3" ) 1174 AT_type ( {0x00000147} ( int ) ) 1175 AT_APPLE_property ( {0x000003cd} ) 1176 AT_artificial ( 0x1 ) 1177 1178New DWARF Tags 1179^^^^^^^^^^^^^^ 1180 1181+-----------------------+--------+ 1182| TAG | Value | 1183+=======================+========+ 1184| DW_TAG_APPLE_property | 0x4200 | 1185+-----------------------+--------+ 1186 1187New DWARF Attributes 1188^^^^^^^^^^^^^^^^^^^^ 1189 1190+--------------------------------+--------+-----------+ 1191| Attribute | Value | Classes | 1192+================================+========+===========+ 1193| DW_AT_APPLE_property | 0x3fed | Reference | 1194+--------------------------------+--------+-----------+ 1195| DW_AT_APPLE_property_getter | 0x3fe9 | String | 1196+--------------------------------+--------+-----------+ 1197| DW_AT_APPLE_property_setter | 0x3fea | String | 1198+--------------------------------+--------+-----------+ 1199| DW_AT_APPLE_property_attribute | 0x3feb | Constant | 1200+--------------------------------+--------+-----------+ 1201 1202New DWARF Constants 1203^^^^^^^^^^^^^^^^^^^ 1204 1205+--------------------------------------+-------+ 1206| Name | Value | 1207+======================================+=======+ 1208| DW_APPLE_PROPERTY_readonly | 0x01 | 1209+--------------------------------------+-------+ 1210| DW_APPLE_PROPERTY_getter | 0x02 | 1211+--------------------------------------+-------+ 1212| DW_APPLE_PROPERTY_assign | 0x04 | 1213+--------------------------------------+-------+ 1214| DW_APPLE_PROPERTY_readwrite | 0x08 | 1215+--------------------------------------+-------+ 1216| DW_APPLE_PROPERTY_retain | 0x10 | 1217+--------------------------------------+-------+ 1218| DW_APPLE_PROPERTY_copy | 0x20 | 1219+--------------------------------------+-------+ 1220| DW_APPLE_PROPERTY_nonatomic | 0x40 | 1221+--------------------------------------+-------+ 1222| DW_APPLE_PROPERTY_setter | 0x80 | 1223+--------------------------------------+-------+ 1224| DW_APPLE_PROPERTY_atomic | 0x100 | 1225+--------------------------------------+-------+ 1226| DW_APPLE_PROPERTY_weak | 0x200 | 1227+--------------------------------------+-------+ 1228| DW_APPLE_PROPERTY_strong | 0x400 | 1229+--------------------------------------+-------+ 1230| DW_APPLE_PROPERTY_unsafe_unretained | 0x800 | 1231+--------------------------------------+-------+ 1232| DW_APPLE_PROPERTY_nullability | 0x1000| 1233+--------------------------------------+-------+ 1234| DW_APPLE_PROPERTY_null_resettable | 0x2000| 1235+--------------------------------------+-------+ 1236| DW_APPLE_PROPERTY_class | 0x4000| 1237+--------------------------------------+-------+ 1238 1239Name Accelerator Tables 1240----------------------- 1241 1242Introduction 1243^^^^^^^^^^^^ 1244 1245The "``.debug_pubnames``" and "``.debug_pubtypes``" formats are not what a 1246debugger needs. The "``pub``" in the section name indicates that the entries 1247in the table are publicly visible names only. This means no static or hidden 1248functions show up in the "``.debug_pubnames``". No static variables or private 1249class variables are in the "``.debug_pubtypes``". Many compilers add different 1250things to these tables, so we can't rely upon the contents between gcc, icc, or 1251clang. 1252 1253The typical query given by users tends not to match up with the contents of 1254these tables. For example, the DWARF spec states that "In the case of the name 1255of a function member or static data member of a C++ structure, class or union, 1256the name presented in the "``.debug_pubnames``" section is not the simple name 1257given by the ``DW_AT_name attribute`` of the referenced debugging information 1258entry, but rather the fully qualified name of the data or function member." 1259So the only names in these tables for complex C++ entries is a fully 1260qualified name. Debugger users tend not to enter their search strings as 1261"``a::b::c(int,const Foo&) const``", but rather as "``c``", "``b::c``" , or 1262"``a::b::c``". So the name entered in the name table must be demangled in 1263order to chop it up appropriately and additional names must be manually entered 1264into the table to make it effective as a name lookup table for debuggers to 1265use. 1266 1267All debuggers currently ignore the "``.debug_pubnames``" table as a result of 1268its inconsistent and useless public-only name content making it a waste of 1269space in the object file. These tables, when they are written to disk, are not 1270sorted in any way, leaving every debugger to do its own parsing and sorting. 1271These tables also include an inlined copy of the string values in the table 1272itself making the tables much larger than they need to be on disk, especially 1273for large C++ programs. 1274 1275Can't we just fix the sections by adding all of the names we need to this 1276table? No, because that is not what the tables are defined to contain and we 1277won't know the difference between the old bad tables and the new good tables. 1278At best we could make our own renamed sections that contain all of the data we 1279need. 1280 1281These tables are also insufficient for what a debugger like LLDB needs. LLDB 1282uses clang for its expression parsing where LLDB acts as a PCH. LLDB is then 1283often asked to look for type "``foo``" or namespace "``bar``", or list items in 1284namespace "``baz``". Namespaces are not included in the pubnames or pubtypes 1285tables. Since clang asks a lot of questions when it is parsing an expression, 1286we need to be very fast when looking up names, as it happens a lot. Having new 1287accelerator tables that are optimized for very quick lookups will benefit this 1288type of debugging experience greatly. 1289 1290We would like to generate name lookup tables that can be mapped into memory 1291from disk, and used as is, with little or no up-front parsing. We would also 1292be able to control the exact content of these different tables so they contain 1293exactly what we need. The Name Accelerator Tables were designed to fix these 1294issues. In order to solve these issues we need to: 1295 1296* Have a format that can be mapped into memory from disk and used as is 1297* Lookups should be very fast 1298* Extensible table format so these tables can be made by many producers 1299* Contain all of the names needed for typical lookups out of the box 1300* Strict rules for the contents of tables 1301 1302Table size is important and the accelerator table format should allow the reuse 1303of strings from common string tables so the strings for the names are not 1304duplicated. We also want to make sure the table is ready to be used as-is by 1305simply mapping the table into memory with minimal header parsing. 1306 1307The name lookups need to be fast and optimized for the kinds of lookups that 1308debuggers tend to do. Optimally we would like to touch as few parts of the 1309mapped table as possible when doing a name lookup and be able to quickly find 1310the name entry we are looking for, or discover there are no matches. In the 1311case of debuggers we optimized for lookups that fail most of the time. 1312 1313Each table that is defined should have strict rules on exactly what is in the 1314accelerator tables and documented so clients can rely on the content. 1315 1316Hash Tables 1317^^^^^^^^^^^ 1318 1319Standard Hash Tables 1320"""""""""""""""""""" 1321 1322Typical hash tables have a header, buckets, and each bucket points to the 1323bucket contents: 1324 1325.. code-block:: none 1326 1327 .------------. 1328 | HEADER | 1329 |------------| 1330 | BUCKETS | 1331 |------------| 1332 | DATA | 1333 `------------' 1334 1335The BUCKETS are an array of offsets to DATA for each hash: 1336 1337.. code-block:: none 1338 1339 .------------. 1340 | 0x00001000 | BUCKETS[0] 1341 | 0x00002000 | BUCKETS[1] 1342 | 0x00002200 | BUCKETS[2] 1343 | 0x000034f0 | BUCKETS[3] 1344 | | ... 1345 | 0xXXXXXXXX | BUCKETS[n_buckets] 1346 '------------' 1347 1348So for ``bucket[3]`` in the example above, we have an offset into the table 13490x000034f0 which points to a chain of entries for the bucket. Each bucket must 1350contain a next pointer, full 32 bit hash value, the string itself, and the data 1351for the current string value. 1352 1353.. code-block:: none 1354 1355 .------------. 1356 0x000034f0: | 0x00003500 | next pointer 1357 | 0x12345678 | 32 bit hash 1358 | "erase" | string value 1359 | data[n] | HashData for this bucket 1360 |------------| 1361 0x00003500: | 0x00003550 | next pointer 1362 | 0x29273623 | 32 bit hash 1363 | "dump" | string value 1364 | data[n] | HashData for this bucket 1365 |------------| 1366 0x00003550: | 0x00000000 | next pointer 1367 | 0x82638293 | 32 bit hash 1368 | "main" | string value 1369 | data[n] | HashData for this bucket 1370 `------------' 1371 1372The problem with this layout for debuggers is that we need to optimize for the 1373negative lookup case where the symbol we're searching for is not present. So 1374if we were to lookup "``printf``" in the table above, we would make a 32-bit 1375hash for "``printf``", it might match ``bucket[3]``. We would need to go to 1376the offset 0x000034f0 and start looking to see if our 32 bit hash matches. To 1377do so, we need to read the next pointer, then read the hash, compare it, and 1378skip to the next bucket. Each time we are skipping many bytes in memory and 1379touching new pages just to do the compare on the full 32 bit hash. All of 1380these accesses then tell us that we didn't have a match. 1381 1382Name Hash Tables 1383"""""""""""""""" 1384 1385To solve the issues mentioned above we have structured the hash tables a bit 1386differently: a header, buckets, an array of all unique 32 bit hash values, 1387followed by an array of hash value data offsets, one for each hash value, then 1388the data for all hash values: 1389 1390.. code-block:: none 1391 1392 .-------------. 1393 | HEADER | 1394 |-------------| 1395 | BUCKETS | 1396 |-------------| 1397 | HASHES | 1398 |-------------| 1399 | OFFSETS | 1400 |-------------| 1401 | DATA | 1402 `-------------' 1403 1404The ``BUCKETS`` in the name tables are an index into the ``HASHES`` array. By 1405making all of the full 32 bit hash values contiguous in memory, we allow 1406ourselves to efficiently check for a match while touching as little memory as 1407possible. Most often checking the 32 bit hash values is as far as the lookup 1408goes. If it does match, it usually is a match with no collisions. So for a 1409table with "``n_buckets``" buckets, and "``n_hashes``" unique 32 bit hash 1410values, we can clarify the contents of the ``BUCKETS``, ``HASHES`` and 1411``OFFSETS`` as: 1412 1413.. code-block:: none 1414 1415 .-------------------------. 1416 | HEADER.magic | uint32_t 1417 | HEADER.version | uint16_t 1418 | HEADER.hash_function | uint16_t 1419 | HEADER.bucket_count | uint32_t 1420 | HEADER.hashes_count | uint32_t 1421 | HEADER.header_data_len | uint32_t 1422 | HEADER_DATA | HeaderData 1423 |-------------------------| 1424 | BUCKETS | uint32_t[n_buckets] // 32 bit hash indexes 1425 |-------------------------| 1426 | HASHES | uint32_t[n_hashes] // 32 bit hash values 1427 |-------------------------| 1428 | OFFSETS | uint32_t[n_hashes] // 32 bit offsets to hash value data 1429 |-------------------------| 1430 | ALL HASH DATA | 1431 `-------------------------' 1432 1433So taking the exact same data from the standard hash example above we end up 1434with: 1435 1436.. code-block:: none 1437 1438 .------------. 1439 | HEADER | 1440 |------------| 1441 | 0 | BUCKETS[0] 1442 | 2 | BUCKETS[1] 1443 | 5 | BUCKETS[2] 1444 | 6 | BUCKETS[3] 1445 | | ... 1446 | ... | BUCKETS[n_buckets] 1447 |------------| 1448 | 0x........ | HASHES[0] 1449 | 0x........ | HASHES[1] 1450 | 0x........ | HASHES[2] 1451 | 0x........ | HASHES[3] 1452 | 0x........ | HASHES[4] 1453 | 0x........ | HASHES[5] 1454 | 0x12345678 | HASHES[6] hash for BUCKETS[3] 1455 | 0x29273623 | HASHES[7] hash for BUCKETS[3] 1456 | 0x82638293 | HASHES[8] hash for BUCKETS[3] 1457 | 0x........ | HASHES[9] 1458 | 0x........ | HASHES[10] 1459 | 0x........ | HASHES[11] 1460 | 0x........ | HASHES[12] 1461 | 0x........ | HASHES[13] 1462 | 0x........ | HASHES[n_hashes] 1463 |------------| 1464 | 0x........ | OFFSETS[0] 1465 | 0x........ | OFFSETS[1] 1466 | 0x........ | OFFSETS[2] 1467 | 0x........ | OFFSETS[3] 1468 | 0x........ | OFFSETS[4] 1469 | 0x........ | OFFSETS[5] 1470 | 0x000034f0 | OFFSETS[6] offset for BUCKETS[3] 1471 | 0x00003500 | OFFSETS[7] offset for BUCKETS[3] 1472 | 0x00003550 | OFFSETS[8] offset for BUCKETS[3] 1473 | 0x........ | OFFSETS[9] 1474 | 0x........ | OFFSETS[10] 1475 | 0x........ | OFFSETS[11] 1476 | 0x........ | OFFSETS[12] 1477 | 0x........ | OFFSETS[13] 1478 | 0x........ | OFFSETS[n_hashes] 1479 |------------| 1480 | | 1481 | | 1482 | | 1483 | | 1484 | | 1485 |------------| 1486 0x000034f0: | 0x00001203 | .debug_str ("erase") 1487 | 0x00000004 | A 32 bit array count - number of HashData with name "erase" 1488 | 0x........ | HashData[0] 1489 | 0x........ | HashData[1] 1490 | 0x........ | HashData[2] 1491 | 0x........ | HashData[3] 1492 | 0x00000000 | String offset into .debug_str (terminate data for hash) 1493 |------------| 1494 0x00003500: | 0x00001203 | String offset into .debug_str ("collision") 1495 | 0x00000002 | A 32 bit array count - number of HashData with name "collision" 1496 | 0x........ | HashData[0] 1497 | 0x........ | HashData[1] 1498 | 0x00001203 | String offset into .debug_str ("dump") 1499 | 0x00000003 | A 32 bit array count - number of HashData with name "dump" 1500 | 0x........ | HashData[0] 1501 | 0x........ | HashData[1] 1502 | 0x........ | HashData[2] 1503 | 0x00000000 | String offset into .debug_str (terminate data for hash) 1504 |------------| 1505 0x00003550: | 0x00001203 | String offset into .debug_str ("main") 1506 | 0x00000009 | A 32 bit array count - number of HashData with name "main" 1507 | 0x........ | HashData[0] 1508 | 0x........ | HashData[1] 1509 | 0x........ | HashData[2] 1510 | 0x........ | HashData[3] 1511 | 0x........ | HashData[4] 1512 | 0x........ | HashData[5] 1513 | 0x........ | HashData[6] 1514 | 0x........ | HashData[7] 1515 | 0x........ | HashData[8] 1516 | 0x00000000 | String offset into .debug_str (terminate data for hash) 1517 `------------' 1518 1519So we still have all of the same data, we just organize it more efficiently for 1520debugger lookup. If we repeat the same "``printf``" lookup from above, we 1521would hash "``printf``" and find it matches ``BUCKETS[3]`` by taking the 32 bit 1522hash value and modulo it by ``n_buckets``. ``BUCKETS[3]`` contains "6" which 1523is the index into the ``HASHES`` table. We would then compare any consecutive 152432 bit hashes values in the ``HASHES`` array as long as the hashes would be in 1525``BUCKETS[3]``. We do this by verifying that each subsequent hash value modulo 1526``n_buckets`` is still 3. In the case of a failed lookup we would access the 1527memory for ``BUCKETS[3]``, and then compare a few consecutive 32 bit hashes 1528before we know that we have no match. We don't end up marching through 1529multiple words of memory and we really keep the number of processor data cache 1530lines being accessed as small as possible. 1531 1532The string hash that is used for these lookup tables is the Daniel J. 1533Bernstein hash which is also used in the ELF ``GNU_HASH`` sections. It is a 1534very good hash for all kinds of names in programs with very few hash 1535collisions. 1536 1537Empty buckets are designated by using an invalid hash index of ``UINT32_MAX``. 1538 1539Details 1540^^^^^^^ 1541 1542These name hash tables are designed to be generic where specializations of the 1543table get to define additional data that goes into the header ("``HeaderData``"), 1544how the string value is stored ("``KeyType``") and the content of the data for each 1545hash value. 1546 1547Header Layout 1548""""""""""""" 1549 1550The header has a fixed part, and the specialized part. The exact format of the 1551header is: 1552 1553.. code-block:: c 1554 1555 struct Header 1556 { 1557 uint32_t magic; // 'HASH' magic value to allow endian detection 1558 uint16_t version; // Version number 1559 uint16_t hash_function; // The hash function enumeration that was used 1560 uint32_t bucket_count; // The number of buckets in this hash table 1561 uint32_t hashes_count; // The total number of unique hash values and hash data offsets in this table 1562 uint32_t header_data_len; // The bytes to skip to get to the hash indexes (buckets) for correct alignment 1563 // Specifically the length of the following HeaderData field - this does not 1564 // include the size of the preceding fields 1565 HeaderData header_data; // Implementation specific header data 1566 }; 1567 1568The header starts with a 32 bit "``magic``" value which must be ``'HASH'`` 1569encoded as an ASCII integer. This allows the detection of the start of the 1570hash table and also allows the table's byte order to be determined so the table 1571can be correctly extracted. The "``magic``" value is followed by a 16 bit 1572``version`` number which allows the table to be revised and modified in the 1573future. The current version number is 1. ``hash_function`` is a ``uint16_t`` 1574enumeration that specifies which hash function was used to produce this table. 1575The current values for the hash function enumerations include: 1576 1577.. code-block:: c 1578 1579 enum HashFunctionType 1580 { 1581 eHashFunctionDJB = 0u, // Daniel J Bernstein hash function 1582 }; 1583 1584``bucket_count`` is a 32 bit unsigned integer that represents how many buckets 1585are in the ``BUCKETS`` array. ``hashes_count`` is the number of unique 32 bit 1586hash values that are in the ``HASHES`` array, and is the same number of offsets 1587are contained in the ``OFFSETS`` array. ``header_data_len`` specifies the size 1588in bytes of the ``HeaderData`` that is filled in by specialized versions of 1589this table. 1590 1591Fixed Lookup 1592"""""""""""" 1593 1594The header is followed by the buckets, hashes, offsets, and hash value data. 1595 1596.. code-block:: c 1597 1598 struct FixedTable 1599 { 1600 uint32_t buckets[Header.bucket_count]; // An array of hash indexes into the "hashes[]" array below 1601 uint32_t hashes [Header.hashes_count]; // Every unique 32 bit hash for the entire table is in this table 1602 uint32_t offsets[Header.hashes_count]; // An offset that corresponds to each item in the "hashes[]" array above 1603 }; 1604 1605``buckets`` is an array of 32 bit indexes into the ``hashes`` array. The 1606``hashes`` array contains all of the 32 bit hash values for all names in the 1607hash table. Each hash in the ``hashes`` table has an offset in the ``offsets`` 1608array that points to the data for the hash value. 1609 1610This table setup makes it very easy to repurpose these tables to contain 1611different data, while keeping the lookup mechanism the same for all tables. 1612This layout also makes it possible to save the table to disk and map it in 1613later and do very efficient name lookups with little or no parsing. 1614 1615DWARF lookup tables can be implemented in a variety of ways and can store a lot 1616of information for each name. We want to make the DWARF tables extensible and 1617able to store the data efficiently so we have used some of the DWARF features 1618that enable efficient data storage to define exactly what kind of data we store 1619for each name. 1620 1621The ``HeaderData`` contains a definition of the contents of each HashData chunk. 1622We might want to store an offset to all of the debug information entries (DIEs) 1623for each name. To keep things extensible, we create a list of items, or 1624Atoms, that are contained in the data for each name. First comes the type of 1625the data in each atom: 1626 1627.. code-block:: c 1628 1629 enum AtomType 1630 { 1631 eAtomTypeNULL = 0u, 1632 eAtomTypeDIEOffset = 1u, // DIE offset, check form for encoding 1633 eAtomTypeCUOffset = 2u, // DIE offset of the compiler unit header that contains the item in question 1634 eAtomTypeTag = 3u, // DW_TAG_xxx value, should be encoded as DW_FORM_data1 (if no tags exceed 255) or DW_FORM_data2 1635 eAtomTypeNameFlags = 4u, // Flags from enum NameFlags 1636 eAtomTypeTypeFlags = 5u, // Flags from enum TypeFlags 1637 }; 1638 1639The enumeration values and their meanings are: 1640 1641.. code-block:: none 1642 1643 eAtomTypeNULL - a termination atom that specifies the end of the atom list 1644 eAtomTypeDIEOffset - an offset into the .debug_info section for the DWARF DIE for this name 1645 eAtomTypeCUOffset - an offset into the .debug_info section for the CU that contains the DIE 1646 eAtomTypeDIETag - The DW_TAG_XXX enumeration value so you don't have to parse the DWARF to see what it is 1647 eAtomTypeNameFlags - Flags for functions and global variables (isFunction, isInlined, isExternal...) 1648 eAtomTypeTypeFlags - Flags for types (isCXXClass, isObjCClass, ...) 1649 1650Then we allow each atom type to define the atom type and how the data for each 1651atom type data is encoded: 1652 1653.. code-block:: c 1654 1655 struct Atom 1656 { 1657 uint16_t type; // AtomType enum value 1658 uint16_t form; // DWARF DW_FORM_XXX defines 1659 }; 1660 1661The ``form`` type above is from the DWARF specification and defines the exact 1662encoding of the data for the Atom type. See the DWARF specification for the 1663``DW_FORM_`` definitions. 1664 1665.. code-block:: c 1666 1667 struct HeaderData 1668 { 1669 uint32_t die_offset_base; 1670 uint32_t atom_count; 1671 Atoms atoms[atom_count0]; 1672 }; 1673 1674``HeaderData`` defines the base DIE offset that should be added to any atoms 1675that are encoded using the ``DW_FORM_ref1``, ``DW_FORM_ref2``, 1676``DW_FORM_ref4``, ``DW_FORM_ref8`` or ``DW_FORM_ref_udata``. It also defines 1677what is contained in each ``HashData`` object -- ``Atom.form`` tells us how large 1678each field will be in the ``HashData`` and the ``Atom.type`` tells us how this data 1679should be interpreted. 1680 1681For the current implementations of the "``.apple_names``" (all functions + 1682globals), the "``.apple_types``" (names of all types that are defined), and 1683the "``.apple_namespaces``" (all namespaces), we currently set the ``Atom`` 1684array to be: 1685 1686.. code-block:: c 1687 1688 HeaderData.atom_count = 1; 1689 HeaderData.atoms[0].type = eAtomTypeDIEOffset; 1690 HeaderData.atoms[0].form = DW_FORM_data4; 1691 1692This defines the contents to be the DIE offset (eAtomTypeDIEOffset) that is 1693encoded as a 32 bit value (DW_FORM_data4). This allows a single name to have 1694multiple matching DIEs in a single file, which could come up with an inlined 1695function for instance. Future tables could include more information about the 1696DIE such as flags indicating if the DIE is a function, method, block, 1697or inlined. 1698 1699The KeyType for the DWARF table is a 32 bit string table offset into the 1700".debug_str" table. The ".debug_str" is the string table for the DWARF which 1701may already contain copies of all of the strings. This helps make sure, with 1702help from the compiler, that we reuse the strings between all of the DWARF 1703sections and keeps the hash table size down. Another benefit to having the 1704compiler generate all strings as DW_FORM_strp in the debug info, is that 1705DWARF parsing can be made much faster. 1706 1707After a lookup is made, we get an offset into the hash data. The hash data 1708needs to be able to deal with 32 bit hash collisions, so the chunk of data 1709at the offset in the hash data consists of a triple: 1710 1711.. code-block:: c 1712 1713 uint32_t str_offset 1714 uint32_t hash_data_count 1715 HashData[hash_data_count] 1716 1717If "str_offset" is zero, then the bucket contents are done. 99.9% of the 1718hash data chunks contain a single item (no 32 bit hash collision): 1719 1720.. code-block:: none 1721 1722 .------------. 1723 | 0x00001023 | uint32_t KeyType (.debug_str[0x0001023] => "main") 1724 | 0x00000004 | uint32_t HashData count 1725 | 0x........ | uint32_t HashData[0] DIE offset 1726 | 0x........ | uint32_t HashData[1] DIE offset 1727 | 0x........ | uint32_t HashData[2] DIE offset 1728 | 0x........ | uint32_t HashData[3] DIE offset 1729 | 0x00000000 | uint32_t KeyType (end of hash chain) 1730 `------------' 1731 1732If there are collisions, you will have multiple valid string offsets: 1733 1734.. code-block:: none 1735 1736 .------------. 1737 | 0x00001023 | uint32_t KeyType (.debug_str[0x0001023] => "main") 1738 | 0x00000004 | uint32_t HashData count 1739 | 0x........ | uint32_t HashData[0] DIE offset 1740 | 0x........ | uint32_t HashData[1] DIE offset 1741 | 0x........ | uint32_t HashData[2] DIE offset 1742 | 0x........ | uint32_t HashData[3] DIE offset 1743 | 0x00002023 | uint32_t KeyType (.debug_str[0x0002023] => "print") 1744 | 0x00000002 | uint32_t HashData count 1745 | 0x........ | uint32_t HashData[0] DIE offset 1746 | 0x........ | uint32_t HashData[1] DIE offset 1747 | 0x00000000 | uint32_t KeyType (end of hash chain) 1748 `------------' 1749 1750Current testing with real world C++ binaries has shown that there is around 1 175132 bit hash collision per 100,000 name entries. 1752 1753Contents 1754^^^^^^^^ 1755 1756As we said, we want to strictly define exactly what is included in the 1757different tables. For DWARF, we have 3 tables: "``.apple_names``", 1758"``.apple_types``", and "``.apple_namespaces``". 1759 1760"``.apple_names``" sections should contain an entry for each DWARF DIE whose 1761``DW_TAG`` is a ``DW_TAG_label``, ``DW_TAG_inlined_subroutine``, or 1762``DW_TAG_subprogram`` that has address attributes: ``DW_AT_low_pc``, 1763``DW_AT_high_pc``, ``DW_AT_ranges`` or ``DW_AT_entry_pc``. It also contains 1764``DW_TAG_variable`` DIEs that have a ``DW_OP_addr`` in the location (global and 1765static variables). All global and static variables should be included, 1766including those scoped within functions and classes. For example using the 1767following code: 1768 1769.. code-block:: c 1770 1771 static int var = 0; 1772 1773 void f () 1774 { 1775 static int var = 0; 1776 } 1777 1778Both of the static ``var`` variables would be included in the table. All 1779functions should emit both their full names and their basenames. For C or C++, 1780the full name is the mangled name (if available) which is usually in the 1781``DW_AT_MIPS_linkage_name`` attribute, and the ``DW_AT_name`` contains the 1782function basename. If global or static variables have a mangled name in a 1783``DW_AT_MIPS_linkage_name`` attribute, this should be emitted along with the 1784simple name found in the ``DW_AT_name`` attribute. 1785 1786"``.apple_types``" sections should contain an entry for each DWARF DIE whose 1787tag is one of: 1788 1789* DW_TAG_array_type 1790* DW_TAG_class_type 1791* DW_TAG_enumeration_type 1792* DW_TAG_pointer_type 1793* DW_TAG_reference_type 1794* DW_TAG_string_type 1795* DW_TAG_structure_type 1796* DW_TAG_subroutine_type 1797* DW_TAG_typedef 1798* DW_TAG_union_type 1799* DW_TAG_ptr_to_member_type 1800* DW_TAG_set_type 1801* DW_TAG_subrange_type 1802* DW_TAG_base_type 1803* DW_TAG_const_type 1804* DW_TAG_file_type 1805* DW_TAG_namelist 1806* DW_TAG_packed_type 1807* DW_TAG_volatile_type 1808* DW_TAG_restrict_type 1809* DW_TAG_atomic_type 1810* DW_TAG_interface_type 1811* DW_TAG_unspecified_type 1812* DW_TAG_shared_type 1813 1814Only entries with a ``DW_AT_name`` attribute are included, and the entry must 1815not be a forward declaration (``DW_AT_declaration`` attribute with a non-zero 1816value). For example, using the following code: 1817 1818.. code-block:: c 1819 1820 int main () 1821 { 1822 int *b = 0; 1823 return *b; 1824 } 1825 1826We get a few type DIEs: 1827 1828.. code-block:: none 1829 1830 0x00000067: TAG_base_type [5] 1831 AT_encoding( DW_ATE_signed ) 1832 AT_name( "int" ) 1833 AT_byte_size( 0x04 ) 1834 1835 0x0000006e: TAG_pointer_type [6] 1836 AT_type( {0x00000067} ( int ) ) 1837 AT_byte_size( 0x08 ) 1838 1839The DW_TAG_pointer_type is not included because it does not have a ``DW_AT_name``. 1840 1841"``.apple_namespaces``" section should contain all ``DW_TAG_namespace`` DIEs. 1842If we run into a namespace that has no name this is an anonymous namespace, and 1843the name should be output as "``(anonymous namespace)``" (without the quotes). 1844Why? This matches the output of the ``abi::cxa_demangle()`` that is in the 1845standard C++ library that demangles mangled names. 1846 1847 1848Language Extensions and File Format Changes 1849^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1850 1851Objective-C Extensions 1852"""""""""""""""""""""" 1853 1854"``.apple_objc``" section should contain all ``DW_TAG_subprogram`` DIEs for an 1855Objective-C class. The name used in the hash table is the name of the 1856Objective-C class itself. If the Objective-C class has a category, then an 1857entry is made for both the class name without the category, and for the class 1858name with the category. So if we have a DIE at offset 0x1234 with a name of 1859method "``-[NSString(my_additions) stringWithSpecialString:]``", we would add 1860an entry for "``NSString``" that points to DIE 0x1234, and an entry for 1861"``NSString(my_additions)``" that points to 0x1234. This allows us to quickly 1862track down all Objective-C methods for an Objective-C class when doing 1863expressions. It is needed because of the dynamic nature of Objective-C where 1864anyone can add methods to a class. The DWARF for Objective-C methods is also 1865emitted differently from C++ classes where the methods are not usually 1866contained in the class definition, they are scattered about across one or more 1867compile units. Categories can also be defined in different shared libraries. 1868So we need to be able to quickly find all of the methods and class functions 1869given the Objective-C class name, or quickly find all methods and class 1870functions for a class + category name. This table does not contain any 1871selector names, it just maps Objective-C class names (or class names + 1872category) to all of the methods and class functions. The selectors are added 1873as function basenames in the "``.debug_names``" section. 1874 1875In the "``.apple_names``" section for Objective-C functions, the full name is 1876the entire function name with the brackets ("``-[NSString 1877stringWithCString:]``") and the basename is the selector only 1878("``stringWithCString:``"). 1879 1880Mach-O Changes 1881"""""""""""""" 1882 1883The sections names for the apple hash tables are for non-mach-o files. For 1884mach-o files, the sections should be contained in the ``__DWARF`` segment with 1885names as follows: 1886 1887* "``.apple_names``" -> "``__apple_names``" 1888* "``.apple_types``" -> "``__apple_types``" 1889* "``.apple_namespaces``" -> "``__apple_namespac``" (16 character limit) 1890* "``.apple_objc``" -> "``__apple_objc``" 1891 1892.. _codeview: 1893 1894CodeView Debug Info Format 1895========================== 1896 1897LLVM supports emitting CodeView, the Microsoft debug info format, and this 1898section describes the design and implementation of that support. 1899 1900Format Background 1901----------------- 1902 1903CodeView as a format is clearly oriented around C++ debugging, and in C++, the 1904majority of debug information tends to be type information. Therefore, the 1905overriding design constraint of CodeView is the separation of type information 1906from other "symbol" information so that type information can be efficiently 1907merged across translation units. Both type information and symbol information is 1908generally stored as a sequence of records, where each record begins with a 190916-bit record size and a 16-bit record kind. 1910 1911Type information is usually stored in the ``.debug$T`` section of the object 1912file. All other debug info, such as line info, string table, symbol info, and 1913inlinee info, is stored in one or more ``.debug$S`` sections. There may only be 1914one ``.debug$T`` section per object file, since all other debug info refers to 1915it. If a PDB (enabled by the ``/Zi`` MSVC option) was used during compilation, 1916the ``.debug$T`` section will contain only an ``LF_TYPESERVER2`` record pointing 1917to the PDB. When using PDBs, symbol information appears to remain in the object 1918file ``.debug$S`` sections. 1919 1920Type records are referred to by their index, which is the number of records in 1921the stream before a given record plus ``0x1000``. Many common basic types, such 1922as the basic integral types and unqualified pointers to them, are represented 1923using type indices less than ``0x1000``. Such basic types are built in to 1924CodeView consumers and do not require type records. 1925 1926Each type record may only contain type indices that are less than its own type 1927index. This ensures that the graph of type stream references is acyclic. While 1928the source-level type graph may contain cycles through pointer types (consider a 1929linked list struct), these cycles are removed from the type stream by always 1930referring to the forward declaration record of user-defined record types. Only 1931"symbol" records in the ``.debug$S`` streams may refer to complete, 1932non-forward-declaration type records. 1933 1934Working with CodeView 1935--------------------- 1936 1937These are instructions for some common tasks for developers working to improve 1938LLVM's CodeView support. Most of them revolve around using the CodeView dumper 1939embedded in ``llvm-readobj``. 1940 1941* Testing MSVC's output:: 1942 1943 $ cl -c -Z7 foo.cpp # Use /Z7 to keep types in the object file 1944 $ llvm-readobj --codeview foo.obj 1945 1946* Getting LLVM IR debug info out of Clang:: 1947 1948 $ clang -g -gcodeview --target=x86_64-windows-msvc foo.cpp -S -emit-llvm 1949 1950 Use this to generate LLVM IR for LLVM test cases. 1951 1952* Generate and dump CodeView from LLVM IR metadata:: 1953 1954 $ llc foo.ll -filetype=obj -o foo.obj 1955 $ llvm-readobj --codeview foo.obj > foo.txt 1956 1957 Use this pattern in lit test cases and FileCheck the output of llvm-readobj 1958 1959Improving LLVM's CodeView support is a process of finding interesting type 1960records, constructing a C++ test case that makes MSVC emit those records, 1961dumping the records, understanding them, and then generating equivalent records 1962in LLVM's backend. 1963 1964Testing Debug Info Preservation in Optimizations 1965================================================ 1966 1967The following paragraphs are an introduction to the debugify utility 1968and examples of how to use it in regression tests to check debug info 1969preservation after optimizations. 1970 1971The ``debugify`` utility 1972------------------------ 1973 1974The ``debugify`` synthetic debug info testing utility consists of two 1975main parts. The ``debugify`` pass and the ``check-debugify`` one. They are 1976meant to be used with ``opt`` for development purposes. 1977 1978The first applies synthetic debug information to every instruction of the module, 1979while the latter checks that this DI is still available after an optimization 1980has occurred, reporting any errors/warnings while doing so. 1981 1982The instructions are assigned sequentially increasing line locations, 1983and are immediately used by debug value intrinsics when possible. 1984 1985For example, here is a module before: 1986 1987.. code-block:: llvm 1988 1989 define void @f(i32* %x) { 1990 entry: 1991 %x.addr = alloca i32*, align 8 1992 store i32* %x, i32** %x.addr, align 8 1993 %0 = load i32*, i32** %x.addr, align 8 1994 store i32 10, i32* %0, align 4 1995 ret void 1996 } 1997 1998and after running ``opt -debugify`` on it we get: 1999 2000.. code-block:: text 2001 2002 define void @f(i32* %x) !dbg !6 { 2003 entry: 2004 %x.addr = alloca i32*, align 8, !dbg !12 2005 call void @llvm.dbg.value(metadata i32** %x.addr, metadata !9, metadata !DIExpression()), !dbg !12 2006 store i32* %x, i32** %x.addr, align 8, !dbg !13 2007 %0 = load i32*, i32** %x.addr, align 8, !dbg !14 2008 call void @llvm.dbg.value(metadata i32* %0, metadata !11, metadata !DIExpression()), !dbg !14 2009 store i32 10, i32* %0, align 4, !dbg !15 2010 ret void, !dbg !16 2011 } 2012 2013 !llvm.dbg.cu = !{!0} 2014 !llvm.debugify = !{!3, !4} 2015 !llvm.module.flags = !{!5} 2016 2017 !0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "debugify", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !2) 2018 !1 = !DIFile(filename: "debugify-sample.ll", directory: "/") 2019 !2 = !{} 2020 !3 = !{i32 5} 2021 !4 = !{i32 2} 2022 !5 = !{i32 2, !"Debug Info Version", i32 3} 2023 !6 = distinct !DISubprogram(name: "f", linkageName: "f", scope: null, file: !1, line: 1, type: !7, isLocal: false, isDefinition: true, scopeLine: 1, isOptimized: true, unit: !0, retainedNodes: !8) 2024 !7 = !DISubroutineType(types: !2) 2025 !8 = !{!9, !11} 2026 !9 = !DILocalVariable(name: "1", scope: !6, file: !1, line: 1, type: !10) 2027 !10 = !DIBasicType(name: "ty64", size: 64, encoding: DW_ATE_unsigned) 2028 !11 = !DILocalVariable(name: "2", scope: !6, file: !1, line: 3, type: !10) 2029 !12 = !DILocation(line: 1, column: 1, scope: !6) 2030 !13 = !DILocation(line: 2, column: 1, scope: !6) 2031 !14 = !DILocation(line: 3, column: 1, scope: !6) 2032 !15 = !DILocation(line: 4, column: 1, scope: !6) 2033 !16 = !DILocation(line: 5, column: 1, scope: !6) 2034 2035The following is an example of the -check-debugify output: 2036 2037.. code-block:: none 2038 2039 $ opt -enable-debugify -loop-vectorize llvm/test/Transforms/LoopVectorize/i8-induction.ll -disable-output 2040 ERROR: Instruction with empty DebugLoc in function f -- %index = phi i32 [ 0, %vector.ph ], [ %index.next, %vector.body ] 2041 2042Errors/warnings can range from instructions with empty debug location to an 2043instruction having a type that's incompatible with the source variable it describes, 2044all the way to missing lines and missing debug value intrinsics. 2045 2046Fixing errors 2047^^^^^^^^^^^^^ 2048 2049Each of the errors above has a relevant API available to fix it. 2050 2051* In the case of missing debug location, ``Instruction::setDebugLoc`` or possibly 2052 ``IRBuilder::setCurrentDebugLocation`` when using a Builder and the new location 2053 should be reused. 2054 2055* When a debug value has incompatible type ``llvm::replaceAllDbgUsesWith`` can be used. 2056 After a RAUW call an incompatible type error can occur because RAUW does not handle 2057 widening and narrowing of variables while ``llvm::replaceAllDbgUsesWith`` does. It is 2058 also capable of changing the DWARF expression used by the debugger to describe the variable. 2059 It also prevents use-before-def by salvaging or deleting invalid debug values. 2060 2061* When a debug value is missing ``llvm::salvageDebugInfo`` can be used when no replacement 2062 exists, or ``llvm::replaceAllDbgUsesWith`` when a replacement exists. 2063 2064Using ``debugify`` 2065------------------ 2066 2067In order for ``check-debugify`` to work, the DI must be coming from 2068``debugify``. Thus, modules with existing DI will be skipped. 2069 2070The most straightforward way to use ``debugify`` is as follows:: 2071 2072 $ opt -debugify -pass-to-test -check-debugify sample.ll 2073 2074This will inject synthetic DI to ``sample.ll`` run the ``pass-to-test`` 2075and then check for missing DI. 2076 2077Some other ways to run debugify are avaliable: 2078 2079.. code-block:: bash 2080 2081 # Same as the above example. 2082 $ opt -enable-debugify -pass-to-test sample.ll 2083 2084 # Suppresses verbose debugify output. 2085 $ opt -enable-debugify -debugify-quiet -pass-to-test sample.ll 2086 2087 # Prepend -debugify before and append -check-debugify -strip after 2088 # each pass on the pipeline (similar to -verify-each). 2089 $ opt -debugify-each -O2 sample.ll 2090 2091``debugify`` can also be used to test a backend, e.g: 2092 2093.. code-block:: bash 2094 2095 $ opt -debugify < sample.ll | llc -o - 2096 2097``debugify`` in regression tests 2098^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2099 2100The ``-debugify`` pass is especially helpful when it comes to testing that 2101a given pass preserves DI while transforming the module. For this to work, 2102the ``-debugify`` output must be stable enough to use in regression tests. 2103Changes to this pass are not allowed to break existing tests. 2104 2105It allows us to test for DI loss in the same tests we check that the 2106transformation is actually doing what it should. 2107 2108Here is an example from ``test/Transforms/InstCombine/cast-mul-select.ll``: 2109 2110.. code-block:: llvm 2111 2112 ; RUN: opt < %s -debugify -instcombine -S | FileCheck %s --check-prefix=DEBUGINFO 2113 2114 define i32 @mul(i32 %x, i32 %y) { 2115 ; DBGINFO-LABEL: @mul( 2116 ; DBGINFO-NEXT: [[C:%.*]] = mul i32 {{.*}} 2117 ; DBGINFO-NEXT: call void @llvm.dbg.value(metadata i32 [[C]] 2118 ; DBGINFO-NEXT: [[D:%.*]] = and i32 {{.*}} 2119 ; DBGINFO-NEXT: call void @llvm.dbg.value(metadata i32 [[D]] 2120 2121 %A = trunc i32 %x to i8 2122 %B = trunc i32 %y to i8 2123 %C = mul i8 %A, %B 2124 %D = zext i8 %C to i32 2125 ret i32 %D 2126 } 2127 2128Here we test that the two ``dbg.value`` instrinsics are preserved and 2129are correctly pointing to the ``[[C]]`` and ``[[D]]`` variables. 2130 2131.. note:: 2132 2133 Note, that when writing this kind of regression tests, it is important 2134 to make them as robust as possible. That's why we should try to avoid 2135 hardcoding line/variable numbers in check lines. If for example you test 2136 for a ``DILocation`` to have a specific line number, and someone later adds 2137 an instruction before the one we check the test will fail. In the cases this 2138 can't be avoided (say, if a test wouldn't be precise enough), moving the 2139 test to its own file is preferred. 2140