1<?xml version="1.0" encoding="ISO-8859-1"?> 2<!DOCTYPE html 3 PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 4 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 5 6<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> 7<head> 8 <meta name="AUTHOR" content="gregod@cs.rpi.edu (Doug Gregor)" /> 9 <meta name="KEYWORDS" content="C++, GCC, libstdc++, g++, debug" /> 10 <meta name="DESCRIPTION" content="Design of the libstdc++ debug mode." /> 11 <meta name="GENERATOR" content="vi and eight fingers" /> 12 <title>Design of the libstdc++ debug mode</title> 13<link rel="StyleSheet" href="lib3styles.css" /> 14</head> 15<body> 16 17<h1 class="centered"><a name="top">Design of the libstdc++ debug mode</a></h1> 18 19<p class="fineprint"><em> 20 The latest version of this document is always available at 21 <a href="http://gcc.gnu.org/onlinedocs/libstdc++/debug_mode.html"> 22 http://gcc.gnu.org/onlinedocs/libstdc++/debug_mode.html</a>. 23</em></p> 24 25<p><em> 26 To the <a href="http://gcc.gnu.org/libstdc++/">libstdc++-v3 homepage</a>. 27</em></p> 28 29 30<!-- ####################################################### --> 31 32<hr /> 33<h1>Debug mode design</h1> 34<p> The libstdc++ debug mode replaces unsafe (but efficient) standard 35 containers and iterators with semantically equivalent safe standard 36 containers and iterators to aid in debugging user programs. The 37 following goals directed the design of the libstdc++ debug mode:</p> 38 39 <ul> 40 41 <li><b>Correctness</b>: the libstdc++ debug mode must not change 42 the semantics of the standard library for all cases specified in 43 the ANSI/ISO C++ standard. The essence of this constraint is that 44 any valid C++ program should behave in the same manner regardless 45 of whether it is compiled with debug mode or release mode. In 46 particular, entities that are defined in namespace std in release 47 mode should remain defined in namespace std in debug mode, so that 48 legal specializations of namespace std entities will remain 49 valid. A program that is not valid C++ (e.g., invokes undefined 50 behavior) is not required to behave similarly, although the debug 51 mode will abort with a diagnostic when it detects undefined 52 behavior.</li> 53 54 <li><b>Performance</b>: the additional of the libstdc++ debug mode 55 must not affect the performance of the library when it is compiled 56 in release mode. Performance of the libstdc++ debug mode is 57 secondary (and, in fact, will be worse than the release 58 mode).</li> 59 60 <li><b>Usability</b>: the libstdc++ debug mode should be easy to 61 use. It should be easily incorporated into the user's development 62 environment (e.g., by requiring only a single new compiler switch) 63 and should produce reasonable diagnostics when it detects a 64 problem with the user program. Usability also involves detection 65 of errors when using the debug mode incorrectly, e.g., by linking 66 a release-compiled object against a debug-compiled object if in 67 fact the resulting program will not run correctly.</li> 68 69 <li><b>Minimize recompilation</b>: While it is expected that 70 users recompile at least part of their program to use debug 71 mode, the amount of recompilation affects the 72 detect-compile-debug turnaround time. This indirectly affects the 73 usefulness of the debug mode, because debugging some applications 74 may require rebuilding a large amount of code, which may not be 75 feasible when the suspect code may be very localized. There are 76 several levels of conformance to this requirement, each with its 77 own usability and implementation characteristics. In general, the 78 higher-numbered conformance levels are more usable (i.e., require 79 less recompilation) but are more complicated to implement than 80 the lower-numbered conformance levels. 81 <ol> 82 <li><b>Full recompilation</b>: The user must recompile his or 83 her entire application and all C++ libraries it depends on, 84 including the C++ standard library that ships with the 85 compiler. This must be done even if only a small part of the 86 program can use debugging features.</li> 87 88 <li><b>Full user recompilation</b>: The user must recompile 89 his or her entire application and all C++ libraries it depends 90 on, but not the C++ standard library itself. This must be done 91 even if only a small part of the program can use debugging 92 features. This can be achieved given a full recompilation 93 system by compiling two versions of the standard library when 94 the compiler is installed and linking against the appropriate 95 one, e.g., a multilibs approach.</li> 96 97 <li><b>Partial recompilation</b>: The user must recompile the 98 parts of his or her application and the C++ libraries it 99 depends on that will use the debugging facilities 100 directly. This means that any code that uses the debuggable 101 standard containers would need to be recompiled, but code 102 that does not use them (but may, for instance, use IOStreams) 103 would not have to be recompiled.</li> 104 105 <li><b>Per-use recompilation</b>: The user must recompile the 106 parts of his or her application and the C++ libraries it 107 depends on where debugging should occur, and any other code 108 that interacts with those containers. This means that a set of 109 translation units that accesses a particular standard 110 container instance may either be compiled in release mode (no 111 checking) or debug mode (full checking), but must all be 112 compiled in the same way; a translation unit that does not see 113 that standard container instance need not be recompiled. This 114 also means that a translation unit <em>A</em> that contains a 115 particular instantiation 116 (say, <code>std::vector<int></code>) compiled in release 117 mode can be linked against a translation unit <em>B</em> that 118 contains the same instantiation compiled in debug mode (a 119 feature not present with partial recompilation). While this 120 behavior is technically a violation of the One Definition 121 Rule, this ability tends to be very important in 122 practice. The libstdc++ debug mode supports this level of 123 recompilation. </li> 124 125 <li><b>Per-unit recompilation</b>: The user must only 126 recompile the translation units where checking should occur, 127 regardless of where debuggable standard containers are 128 used. This has also been dubbed "<code>-g</code> mode", 129 because the <code>-g</code> compiler switch works in this way, 130 emitting debugging information at a per--translation-unit 131 granularity. We believe that this level of recompilation is in 132 fact not possible if we intend to supply safe iterators, leave 133 the program semantics unchanged, and not regress in 134 performance under release mode because we cannot associate 135 extra information with an iterator (to form a safe iterator) 136 without either reserving that space in release mode 137 (performance regression) or allocating extra memory associated 138 with each iterator with <code>new</code> (changes the program 139 semantics).</li> 140 </ol> 141 </li> 142 </ul> 143 144<h2><a name="other">Other implementations</a></h2> 145<p> There are several existing implementations of debug modes for C++ 146 standard library implementations, although none of them directly 147 supports debugging for programs using libstdc++. The existing 148 implementations include:</p> 149<ul> 150 <li><a 151 href="http://www.mathcs.sjsu.edu/faculty/horstman/safestl.html">SafeSTL</a>: 152 SafeSTL was the original debugging version of the Standard Template 153 Library (STL), implemented by Cay S. Horstmann on top of the 154 Hewlett-Packard STL. Though it inspired much work in this area, it 155 has not been kept up-to-date for use with modern compilers or C++ 156 standard library implementations.</li> 157 158 <li><a href="http://www.stlport.org/">STLport</a>: STLport is a free 159 implementation of the C++ standard library derived from the <a 160 href="http://www.sgi.com/tech/stl/">SGI implementation</a>, and 161 ported to many other platforms. It includes a debug mode that uses a 162 wrapper model (that in some way inspired the libstdc++ debug mode 163 design), although at the time of this writing the debug mode is 164 somewhat incomplete and meets only the "Full user recompilation" (2) 165 recompilation guarantee by requiring the user to link against a 166 different library in debug mode vs. release mode.</li> 167 168 <li><a href="http://www.metrowerks.com/mw/default.htm">Metrowerks 169 CodeWarrior</a>: The C++ standard library that ships with Metrowerks 170 CodeWarrior includes a debug mode. It is a full debug-mode 171 implementation (including debugging for CodeWarrior extensions) and 172 is easy to use, although it meets only the "Full recompilation" (1) 173 recompilation guarantee.</li> 174</ul> 175 176<h2><a name="design">Debug mode design methodology</a></h2> 177<p>This section provides an overall view of the design of the 178 libstdc++ debug mode and details the relationship between design 179 decisions and the stated design goals.</p> 180 181<h3><a name="wrappers">The wrapper model</a></h3> 182<p>The libstdc++ debug mode uses a wrapper model where the debugging 183 versions of library components (e.g., iterators and containers) form 184 a layer on top of the release versions of the library 185 components. The debugging components first verify that the operation 186 is correct (aborting with a diagnostic if an error is found) and 187 will then forward to the underlying release-mode container that will 188 perform the actual work. This design decision ensures that we cannot 189 regress release-mode performance (because the release-mode 190 containers are left untouched) and partially enables <a 191 href="#mixing">mixing debug and release code</a> at link time, 192 although that will not be discussed at this time.</p> 193 194<p>Two types of wrappers are used in the implementation of the debug 195 mode: container wrappers and iterator wrappers. The two types of 196 wrappers interact to maintain relationships between iterators and 197 their associated containers, which are necessary to detect certain 198 types of standard library usage errors such as dereferencing 199 past-the-end iterators or inserting into a container using an 200 iterator from a different container.</p> 201 202<h4><a name="safe_iterator">Safe iterators</a></h4> 203<p>Iterator wrappers provide a debugging layer over any iterator that 204 is attached to a particular container, and will manage the 205 information detailing the iterator's state (singular, 206 dereferenceable, etc.) and tracking the container to which the 207 iterator is attached. Because iterators have a well-defined, common 208 interface the iterator wrapper is implemented with the iterator 209 adaptor class template <code>__gnu_debug::_Safe_iterator</code>, 210 which takes two template parameters:</p> 211 212<ul> 213 <li><code>Iterator</code>: The underlying iterator type, which must 214 be either the <code>iterator</code> or <code>const_iterator</code> 215 typedef from the sequence type this iterator can reference.</li> 216 217 <li><code>Sequence</code>: The type of sequence that this iterator 218 references. This sequence must be a safe sequence (discussed below) 219 whose <code>iterator</code> or <code>const_iterator</code> typedef 220 is the type of the safe iterator.</li> 221</ul> 222 223<h4><a name="safe_sequence">Safe sequences (containers)</a></h4> 224<p>Container wrappers provide a debugging layer over a particular 225 container type. Because containers vary greatly in the member 226 functions they support and the semantics of those member functions 227 (especially in the area of iterator invalidation), container 228 wrappers are tailored to the container they reference, e.g., the 229 debugging version of <code>std::list</code> duplicates the entire 230 interface of <code>std::list</code>, adding additional semantic 231 checks and then forwarding operations to the 232 real <code>std::list</code> (a public base class of the debugging 233 version) as appropriate. However, all safe containers inherit from 234 the class template <code>__gnu_debug::_Safe_sequence</code>, 235 instantiated with the type of the safe container itself (an instance 236 of the curiously recurring template pattern).</p> 237 238<p>The iterators of a container wrapper will be 239 <a href="#safe_iterator">safe iterators</a> that reference sequences 240 of this type and wrap the iterators provided by the release-mode 241 base class. The debugging container will use only the safe 242 iterators within its own interface (therefore requiring the user to 243 use safe iterators, although this does not change correct user 244 code) and will communicate with the release-mode base class with 245 only the underlying, unsafe, release-mode iterators that the base 246 class exports.</p> 247 248<p> The debugging version of <code>std::list</code> will have the 249 following basic structure:</p> 250 251<pre> 252template<typename _Tp, typename _Allocator = allocator<_Tp> 253 class debug-list : 254 public release-list<_Tp, _Allocator>, 255 public __gnu_debug::_Safe_sequence<debug-list<_Tp, _Allocator> > 256 { 257 typedef release-list<_Tp, _Allocator> _Base; 258 typedef debug-list<_Tp, _Allocator> _Self; 259 260 public: 261 typedef __gnu_debug::_Safe_iterator<typename _Base::iterator, _Self> iterator; 262 typedef __gnu_debug::_Safe_iterator<typename _Base::const_iterator, _Self> const_iterator; 263 264 // duplicate std::list interface with debugging semantics 265 }; 266</pre> 267 268<h3><a name="precondition">Precondition checking</a></h3> 269<p>The debug mode operates primarily by checking the preconditions of 270 all standard library operations that it supports. Preconditions that 271 are always checked (regardless of whether or not we are in debug 272 mode) are checked via the <code>__check_xxx</code> macros defined 273 and documented in the source 274 file <code>include/debug/debug.h</code>. Preconditions that may or 275 may not be checked, depending on the debug-mode 276 macro <code>_GLIBCXX_DEBUG</code>, are checked via 277 the <code>__requires_xxx</code> macros defined and documented in the 278 same source file. Preconditions are validated using any additional 279 information available at run-time, e.g., the containers that are 280 associated with a particular iterator, the position of the iterator 281 within those containers, the distance between two iterators that may 282 form a valid range, etc. In the absence of suitable information, 283 e.g., an input iterator that is not a safe iterator, these 284 precondition checks will silently succeed.</p> 285 286<p>The majority of precondition checks use the aforementioned macros, 287 which have the secondary benefit of having prewritten debug 288 messages that use information about the current status of the 289 objects involved (e.g., whether an iterator is singular or what 290 sequence it is attached to) along with some static information 291 (e.g., the names of the function parameters corresponding to the 292 objects involved). When not using these macros, the debug mode uses 293 either the debug-mode assertion 294 macro <code>_GLIBCXX_DEBUG_ASSERT</code> , its pedantic 295 cousin <code>_GLIBCXX_DEBUG_PEDASSERT</code>, or the assertion 296 check macro that supports more advance formulation of error 297 messages, <code>_GLIBCXX_DEBUG_VERIFY</code>. These macros are 298 documented more thoroughly in the debug mode source code.</p> 299 300<h3><a name="coexistence">Release- and debug-mode coexistence</a></h3> 301<p>The libstdc++ debug mode is the first debug mode we know of that 302 is able to provide the "Per-use recompilation" (4) guarantee, that 303 allows release-compiled and debug-compiled code to be linked and 304 executed together without causing unpredictable behavior. This 305 guarantee minimizes the recompilation that users are required to 306 perform, shortening the detect-compile-debug bughunting cycle 307 and making the debug mode easier to incorporate into development 308 environments by minimizing dependencies.</p> 309 310<p>Achieving link- and run-time coexistence is not a trivial 311 implementation task. To achieve this goal we required a small 312 extension to the GNU C++ compiler (described in the GCC Manual for 313 C++ Extensions, see <a 314 href="http://gcc.gnu.org/onlinedocs/gcc/Strong-Using.html">strong 315 using</a>), and a complex organization of debug- and 316 release-modes. The end result is that we have achieved per-use 317 recompilation but have had to give up some checking of the 318 <code>std::basic_string</code> class template (namely, safe 319 iterators). 320</p> 321 322<h4><a name="compile_coexistence">Compile-time coexistence of release- and 323 debug-mode components</a></h4> 324<p>Both the release-mode components and the debug-mode 325 components need to exist within a single translation unit so that 326 the debug versions can wrap the release versions. However, only one 327 of these components should be user-visible at any particular 328 time with the standard name, e.g., <code>std::list</code>. </p> 329 330<p>In release mode, we define only the release-mode version of the 331 component with its standard name and do not include the debugging 332 component at all. The release mode version is defined within the 333 namespace <code>std</code>. Minus the namespace associations, this 334 method leaves the behavior of release mode completely unchanged from 335 its behavior prior to the introduction of the libstdc++ debug 336 mode. Here's an example of what this ends up looking like, in 337 C++.</p> 338 339<pre> 340namespace std 341{ 342 template<typename _Tp, typename _Alloc = allocator<_Tp> > 343 class list 344 { 345 // ... 346 }; 347} // namespace std 348</pre> 349 350<p>In debug mode we include the release-mode container (which is now 351defined in in the namespace <code>__norm</code>) and also the 352debug-mode container. The debug-mode container is defined within the 353namespace <code>__debug</code>, which is associated with namespace 354<code>std</code> via the GNU namespace association extension. This 355method allows the debug and release versions of the same component to 356coexist at compile-time and link-time without causing an unreasonable 357maintenance burden, while minimizing confusion. Again, this boils down 358to C++ code as follows:</p> 359 360<pre> 361namespace std 362{ 363 namespace __norm 364 { 365 template<typename _Tp, typename _Alloc = allocator<_Tp> > 366 class list 367 { 368 // ... 369 }; 370 } // namespace __gnu_norm 371 372 namespace __debug 373 { 374 template<typename _Tp, typename _Alloc = allocator<_Tp> > 375 class list 376 : public __norm::list<_Tp, _Alloc>, 377 public __gnu_debug::_Safe_sequence<list<_Tp, _Alloc> > 378 { 379 // ... 380 }; 381 } // namespace __norm 382 383 using namespace __debug __attribute__ ((strong)); 384} 385</pre> 386 387<h4><a name="mixing">Link- and run-time coexistence of release- and 388 debug-mode components</a></h4> 389 390<p>Because each component has a distinct and separate release and 391debug implementation, there are are no issues with link-time 392coexistence: the separate namespaces result in different mangled 393names, and thus unique linkage.</p> 394 395<p>However, components that are defined and used within the C++ 396standard library itself face additional constraints. For instance, 397some of the member functions of <code> std::moneypunct</code> return 398<code>std::basic_string</code>. Normally, this is not a problem, but 399with a mixed mode standard library that could be using either 400debug-mode or release-mode <code> basic_string</code> objects, things 401get more complicated. As the return value of a function is not 402encoded into the mangled name, there is no way to specify a 403release-mode or a debug-mode string. In practice, this results in 404runtime errors. A simplified example of this problem is as follows. 405</p> 406 407<p> Take this translation unit, compiled in debug-mode: </p> 408<pre> 409// -D_GLIBCXX_DEBUG 410#include <string> 411 412std::string test02(); 413 414std::string test01() 415{ 416 return test02(); 417} 418 419int main() 420{ 421 test01(); 422 return 0; 423} 424</pre> 425 426<p> ... and linked to this translation unit, compiled in release mode:</p> 427 428<pre> 429#include <string> 430 431std::string 432test02() 433{ 434 return std::string("toast"); 435} 436</pre> 437 438<p> For this reason we cannot easily provide safe iterators for 439 the <code>std::basic_string</code> class template, as it is present 440 throughout the C++ standard library. For instance, locale facets 441 define typedefs that include <code>basic_string</code>: in a mixed 442 debug/release program, should that typedef be based on the 443 debug-mode <code>basic_string</code> or the 444 release-mode <code>basic_string</code>? While the answer could be 445 "both", and the difference hidden via renaming a la the 446 debug/release containers, we must note two things about locale 447 facets:</p> 448 449<ol> 450 <li>They exist as shared state: one can create a facet in one 451 translation unit and access the facet via the same type name in a 452 different translation unit. This means that we cannot have two 453 different versions of locale facets, because the types would not be 454 the same across debug/release-mode translation unit barriers.</li> 455 456 <li>They have virtual functions returning strings: these functions 457 mangle in the same way regardless of the mangling of their return 458 types (see above), and their precise signatures can be relied upon 459 by users because they may be overridden in derived classes.</li> 460</ol> 461 462<p>With the design of libstdc++ debug mode, we cannot effectively hide 463 the differences between debug and release-mode strings from the 464 user. Failure to hide the differences may result in unpredictable 465 behavior, and for this reason we have opted to only 466 perform <code>basic_string</code> changes that do not require ABI 467 changes. The effect on users is expected to be minimal, as there are 468 simple alternatives (e.g., <code>__gnu_debug::basic_string</code>), 469 and the usability benefit we gain from the ability to mix debug- and 470 release-compiled translation units is enormous.</p> 471 472<h4><a name="coexistence_alt">Alternatives for Coexistence</a></h4> 473<p>The coexistence scheme above was chosen over many alternatives, 474 including language-only solutions and solutions that also required 475 extensions to the C++ front end. The following is a partial list of 476 solutions, with justifications for our rejection of each.</p> 477 478<ul> 479 <li><em>Completely separate debug/release libraries</em>: This is by 480 far the simplest implementation option, where we do not allow any 481 coexistence of debug- and release-compiled translation units in a 482 program. This solution has an extreme negative affect on usability, 483 because it is quite likely that some libraries an application 484 depends on cannot be recompiled easily. This would not meet 485 our <b>usability</b> or <b>minimize recompilation</b> criteria 486 well.</li> 487 488 <li><em>Add a <code>Debug</code> boolean template parameter</em>: 489 Partial specialization could be used to select the debug 490 implementation when <code>Debug == true</code>, and the state 491 of <code>_GLIBCXX_DEBUG</code> could decide whether the 492 default <code>Debug</code> argument is <code>true</code> 493 or <code>false</code>. This option would break conformance with the 494 C++ standard in both debug <em>and</em> release modes. This would 495 not meet our <b>correctness</b> criteria. </li> 496 497 <li><em>Packaging a debug flag in the allocators</em>: We could 498 reuse the <code>Allocator</code> template parameter of containers 499 by adding a sentinel wrapper <code>debug<></code> that 500 signals the user's intention to use debugging, and pick up 501 the <code>debug<></code> allocator wrapper in a partial 502 specialization. However, this has two drawbacks: first, there is a 503 conformance issue because the default allocator would not be the 504 standard-specified <code>std::allocator<T></code>. Secondly 505 (and more importantly), users that specify allocators instead of 506 implicitly using the default allocator would not get debugging 507 containers. Thus this solution fails the <b>correctness</b> 508 criteria.</li> 509 510 <li><em>Define debug containers in another namespace, and employ 511 a <code>using</code> declaration (or directive)</em>: This is an 512 enticing option, because it would eliminate the need for 513 the <code>link_name</code> extension by aliasing the 514 templates. However, there is no true template aliasing mechanism 515 is C++, because both <code>using</code> directives and using 516 declarations disallow specialization. This method fails 517 the <b>correctness</b> criteria.</li> 518 519 <li><em> Use implementation-specific properties of anonymous 520 namespaces. </em> 521 See <a 522 href="http://gcc.gnu.org/ml/libstdc++/2003-08/msg00004.html"> this post 523 </a> 524 This method fails the <b>correctness</b> criteria.</li> 525 526 <li><em>Extension: allow reopening on namespaces</em>: This would 527 allow the debug mode to effectively alias the 528 namespace <code>std</code> to an internal namespace, such 529 as <code>__gnu_std_debug</code>, so that it is completely 530 separate from the release-mode <code>std</code> namespace. While 531 this will solve some renaming problems and ensure that 532 debug- and release-compiled code cannot be mixed unsafely, it ensures that 533 debug- and release-compiled code cannot be mixed at all. For 534 instance, the program would have two <code>std::cout</code> 535 objects! This solution would fails the <b>minimize 536 recompilation</b> requirement, because we would only be able to 537 support option (1) or (2).</li> 538 539 <li><em>Extension: use link name</em>: This option involves 540 complicated re-naming between debug-mode and release-mode 541 components at compile time, and then a g++ extension called <em> 542 link name </em> to recover the original names at link time. There 543 are two drawbacks to this approach. One, it's very verbose, 544 relying on macro renaming at compile time and several levels of 545 include ordering. Two, ODR issues remained with container member 546 functions taking no arguments in mixed-mode settings resulting in 547 equivalent link names, <code> vector::push_back() </code> being 548 one example. 549 See <a 550 href="http://gcc.gnu.org/ml/libstdc++/2003-08/msg00177.html">link 551 name</a> </li> 552</ul> 553 554<p>Other options may exist for implementing the debug mode, many of 555 which have probably been considered and others that may still be 556 lurking. This list may be expanded over time to include other 557 options that we could have implemented, but in all cases the full 558 ramifications of the approach (as measured against the design goals 559 for a libstdc++ debug mode) should be considered first. The DejaGNU 560 testsuite includes some testcases that check for known problems with 561 some solutions (e.g., the <code>using</code> declaration solution 562 that breaks user specialization), and additional testcases will be 563 added as we are able to identify other typical problem cases. These 564 test cases will serve as a benchmark by which we can compare debug 565 mode implementations.</p> 566 567<!-- ####################################################### --> 568 569<hr /> 570<p class="fineprint"><em> 571See <a href="17_intro/license.html">license.html</a> for copying conditions. 572Comments and suggestions are welcome, and may be sent to 573<a href="mailto:libstdc++@gcc.gnu.org">the libstdc++ mailing list</a>. 574</em></p> 575 576 577</body> 578</html> 579