1=============================== 2ORC Design and Implementation 3=============================== 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11This document aims to provide a high-level overview of the design and 12implementation of the ORC JIT APIs. Except where otherwise stated, all 13discussion applies to the design of the APIs as of LLVM Version 10 (ORCv2). 14 15Use-cases 16========= 17 18ORC provides a modular API for building JIT compilers. There are a range 19of use cases for such an API. For example: 20 211. The LLVM tutorials use a simple ORC-based JIT class to execute expressions 22compiled from a toy language: Kaleidoscope. 23 242. The LLVM debugger, LLDB, uses a cross-compiling JIT for expression 25evaluation. In this use case, cross compilation allows expressions compiled 26in the debugger process to be executed on the debug target process, which may 27be on a different device/architecture. 28 293. In high-performance JITs (e.g. JVMs, Julia) that want to make use of LLVM's 30optimizations within an existing JIT infrastructure. 31 324. In interpreters and REPLs, e.g. Cling (C++) and the Swift interpreter. 33 34By adopting a modular, library-based design we aim to make ORC useful in as many 35of these contexts as possible. 36 37Features 38======== 39 40ORC provides the following features: 41 42*JIT-linking* 43 ORC provides APIs to link relocatable object files (COFF, ELF, MachO) [1]_ 44 into a target process at runtime. The target process may be the same process 45 that contains the JIT session object and jit-linker, or may be another process 46 (even one running on a different machine or architecture) that communicates 47 with the JIT via RPC. 48 49*LLVM IR compilation* 50 ORC provides off the shelf components (IRCompileLayer, SimpleCompiler, 51 ConcurrentIRCompiler) that make it easy to add LLVM IR to a JIT'd process. 52 53*Eager and lazy compilation* 54 By default, ORC will compile symbols as soon as they are looked up in the JIT 55 session object (``ExecutionSession``). Compiling eagerly by default makes it 56 easy to use ORC as a simple in-memory compiler within an existing JIT 57 infrastructure. However ORC also provides support for lazy compilation via 58 lazy-reexports (see :ref:`Laziness`). 59 60*Support for Custom Compilers and Program Representations* 61 Clients can supply custom compilers for each symbol that they define in their 62 JIT session. ORC will run the user-supplied compiler when the a definition of 63 a symbol is needed. ORC is actually fully language agnostic: LLVM IR is not 64 treated specially, and is supported via the same wrapper mechanism (the 65 ``MaterializationUnit`` class) that is used for custom compilers. 66 67*Concurrent JIT'd code* and *Concurrent Compilation* 68 JIT'd code may spawn multiple threads, and may re-enter the JIT (e.g. for lazy 69 compilation) concurrently from multiple threads. The ORC APIs also support 70 running multiple compilers concurrently. Built-in dependency tracking (via the 71 JIT linker) ensures that ORC does not release code for execution until it is 72 safe to call. 73 74*Orthogonality* and *Composability* 75 Each of the features above can be used (or not) independently. It is possible 76 to put ORC components together to make a non-lazy, in-process, single threaded 77 JIT or a lazy, out-of-process, concurrent JIT, or anything in between. 78 79LLJIT and LLLazyJIT 80=================== 81 82ORC provides two basic JIT classes off-the-shelf. These are useful both as 83examples of how to assemble ORC components to make a JIT, and as replacements 84for earlier LLVM JIT APIs (e.g. MCJIT). 85 86The LLJIT class uses an IRCompileLayer and RTDyldObjectLinkingLayer to support 87compilation of LLVM IR and linking of relocatable object files. All operations 88are performed eagerly on symbol lookup (i.e. a symbol's definition is compiled 89as soon as you attempt to look up its address). LLJIT is a suitable replacement 90for MCJIT in most cases (note: some more advanced features, e.g. 91JITEventListeners are not supported yet). 92 93The LLLazyJIT extends LLJIT and adds a CompileOnDemandLayer to enable lazy 94compilation of LLVM IR. When an LLVM IR module is added via the addLazyIRModule 95method, function bodies in that module will not be compiled until they are first 96called. LLLazyJIT aims to provide a replacement of LLVM's original (pre-MCJIT) 97JIT API. 98 99LLJIT and LLLazyJIT instances can be created using their respective builder 100classes: LLJITBuilder and LLazyJITBuilder. For example, assuming you have a 101module ``M`` loaded on a ThreadSafeContext ``Ctx``: 102 103.. code-block:: c++ 104 105 // Try to detect the host arch and construct an LLJIT instance. 106 auto JIT = LLJITBuilder().create(); 107 108 // If we could not construct an instance, return an error. 109 if (!JIT) 110 return JIT.takeError(); 111 112 // Add the module. 113 if (auto Err = JIT->addIRModule(TheadSafeModule(std::move(M), Ctx))) 114 return Err; 115 116 // Look up the JIT'd code entry point. 117 auto EntrySym = JIT->lookup("entry"); 118 if (!EntrySym) 119 return EntrySym.takeError(); 120 121 // Cast the entry point address to a function pointer. 122 auto *Entry = (void(*)())EntrySym.getAddress(); 123 124 // Call into JIT'd code. 125 Entry(); 126 127The builder classes provide a number of configuration options that can be 128specified before the JIT instance is constructed. For example: 129 130.. code-block:: c++ 131 132 // Build an LLLazyJIT instance that uses four worker threads for compilation, 133 // and jumps to a specific error handler (rather than null) on lazy compile 134 // failures. 135 136 void handleLazyCompileFailure() { 137 // JIT'd code will jump here if lazy compilation fails, giving us an 138 // opportunity to exit or throw an exception into JIT'd code. 139 throw JITFailed(); 140 } 141 142 auto JIT = LLLazyJITBuilder() 143 .setNumCompileThreads(4) 144 .setLazyCompileFailureAddr( 145 toJITTargetAddress(&handleLazyCompileFailure)) 146 .create(); 147 148 // ... 149 150For users wanting to get started with LLJIT a minimal example program can be 151found at ``llvm/examples/HowToUseLLJIT``. 152 153Design Overview 154=============== 155 156ORC's JIT'd program model aims to emulate the linking and symbol resolution 157rules used by the static and dynamic linkers. This allows ORC to JIT 158arbitrary LLVM IR, including IR produced by an ordinary static compiler (e.g. 159clang) that uses constructs like symbol linkage and visibility, and weak [3]_ 160and common symbol definitions. 161 162To see how this works, imagine a program ``foo`` which links against a pair 163of dynamic libraries: ``libA`` and ``libB``. On the command line, building this 164program might look like: 165 166.. code-block:: bash 167 168 $ clang++ -shared -o libA.dylib a1.cpp a2.cpp 169 $ clang++ -shared -o libB.dylib b1.cpp b2.cpp 170 $ clang++ -o myapp myapp.cpp -L. -lA -lB 171 $ ./myapp 172 173In ORC, this would translate into API calls on a "CXXCompilingLayer" (with error 174checking omitted for brevity) as: 175 176.. code-block:: c++ 177 178 ExecutionSession ES; 179 RTDyldObjectLinkingLayer ObjLinkingLayer( 180 ES, []() { return std::make_unique<SectionMemoryManager>(); }); 181 CXXCompileLayer CXXLayer(ES, ObjLinkingLayer); 182 183 // Create JITDylib "A" and add code to it using the CXX layer. 184 auto &LibA = ES.createJITDylib("A"); 185 CXXLayer.add(LibA, MemoryBuffer::getFile("a1.cpp")); 186 CXXLayer.add(LibA, MemoryBuffer::getFile("a2.cpp")); 187 188 // Create JITDylib "B" and add code to it using the CXX layer. 189 auto &LibB = ES.createJITDylib("B"); 190 CXXLayer.add(LibB, MemoryBuffer::getFile("b1.cpp")); 191 CXXLayer.add(LibB, MemoryBuffer::getFile("b2.cpp")); 192 193 // Create and specify the search order for the main JITDylib. This is 194 // equivalent to a "links against" relationship in a command-line link. 195 auto &MainJD = ES.createJITDylib("main"); 196 MainJD.setSearchOrder({{&LibA, false}, {&LibB, false}}); 197 CXXLayer.add(MainJD, MemoryBuffer::getFile("main.cpp")); 198 199 // Look up the JIT'd main, cast it to a function pointer, then call it. 200 auto MainSym = ExitOnErr(ES.lookup({&MainJD}, "main")); 201 auto *Main = (int(*)(int, char*[]))MainSym.getAddress(); 202 203 int Result = Main(...); 204 205This example tells us nothing about *how* or *when* compilation will happen. 206That will depend on the implementation of the hypothetical CXXCompilingLayer. 207The same linker-based symbol resolution rules will apply regardless of that 208implementation, however. For example, if a1.cpp and a2.cpp both define a 209function "foo" then ORCv2 will generate a duplicate definition error. On the 210other hand, if a1.cpp and b1.cpp both define "foo" there is no error (different 211dynamic libraries may define the same symbol). If main.cpp refers to "foo", it 212should bind to the definition in LibA rather than the one in LibB, since 213main.cpp is part of the "main" dylib, and the main dylib links against LibA 214before LibB. 215 216Many JIT clients will have no need for this strict adherence to the usual 217ahead-of-time linking rules, and should be able to get by just fine by putting 218all of their code in a single JITDylib. However, clients who want to JIT code 219for languages/projects that traditionally rely on ahead-of-time linking (e.g. 220C++) will find that this feature makes life much easier. 221 222Symbol lookup in ORC serves two other important functions, beyond providing 223addresses for symbols: (1) It triggers compilation of the symbol(s) searched for 224(if they have not been compiled already), and (2) it provides the 225synchronization mechanism for concurrent compilation. The pseudo-code for the 226lookup process is: 227 228.. code-block:: none 229 230 construct a query object from a query set and query handler 231 lock the session 232 lodge query against requested symbols, collect required materializers (if any) 233 unlock the session 234 dispatch materializers (if any) 235 236In this context a materializer is something that provides a working definition 237of a symbol upon request. Usually materializers are just wrappers for compilers, 238but they may also wrap a jit-linker directly (if the program representation 239backing the definitions is an object file), or may even be a class that writes 240bits directly into memory (for example, if the definitions are 241stubs). Materialization is the blanket term for any actions (compiling, linking, 242splatting bits, registering with runtimes, etc.) that are required to generate a 243symbol definition that is safe to call or access. 244 245As each materializer completes its work it notifies the JITDylib, which in turn 246notifies any query objects that are waiting on the newly materialized 247definitions. Each query object maintains a count of the number of symbols that 248it is still waiting on, and once this count reaches zero the query object calls 249the query handler with a *SymbolMap* (a map of symbol names to addresses) 250describing the result. If any symbol fails to materialize the query immediately 251calls the query handler with an error. 252 253The collected materialization units are sent to the ExecutionSession to be 254dispatched, and the dispatch behavior can be set by the client. By default each 255materializer is run on the calling thread. Clients are free to create new 256threads to run materializers, or to send the work to a work queue for a thread 257pool (this is what LLJIT/LLLazyJIT do). 258 259Top Level APIs 260============== 261 262Many of ORC's top-level APIs are visible in the example above: 263 264- *ExecutionSession* represents the JIT'd program and provides context for the 265 JIT: It contains the JITDylibs, error reporting mechanisms, and dispatches the 266 materializers. 267 268- *JITDylibs* provide the symbol tables. 269 270- *Layers* (ObjLinkingLayer and CXXLayer) are wrappers around compilers and 271 allow clients to add uncompiled program representations supported by those 272 compilers to JITDylibs. 273 274Several other important APIs are used explicitly. JIT clients need not be aware 275of them, but Layer authors will use them: 276 277- *MaterializationUnit* - When XXXLayer::add is invoked it wraps the given 278 program representation (in this example, C++ source) in a MaterializationUnit, 279 which is then stored in the JITDylib. MaterializationUnits are responsible for 280 describing the definitions they provide, and for unwrapping the program 281 representation and passing it back to the layer when compilation is required 282 (this ownership shuffle makes writing thread-safe layers easier, since the 283 ownership of the program representation will be passed back on the stack, 284 rather than having to be fished out of a Layer member, which would require 285 synchronization). 286 287- *MaterializationResponsibility* - When a MaterializationUnit hands a program 288 representation back to the layer it comes with an associated 289 MaterializationResponsibility object. This object tracks the definitions 290 that must be materialized and provides a way to notify the JITDylib once they 291 are either successfully materialized or a failure occurs. 292 293Absolute Symbols, Aliases, and Reexports 294======================================== 295 296ORC makes it easy to define symbols with absolute addresses, or symbols that 297are simply aliases of other symbols: 298 299Absolute Symbols 300---------------- 301 302Absolute symbols are symbols that map directly to addresses without requiring 303further materialization, for example: "foo" = 0x1234. One use case for 304absolute symbols is allowing resolution of process symbols. E.g. 305 306.. code-block: c++ 307 308 JD.define(absoluteSymbols(SymbolMap({ 309 { Mangle("printf"), 310 { pointerToJITTargetAddress(&printf), 311 JITSymbolFlags::Callable } } 312 }); 313 314With this mapping established code added to the JIT can refer to printf 315symbolically rather than requiring the address of printf to be "baked in". 316This in turn allows cached versions of the JIT'd code (e.g. compiled objects) 317to be re-used across JIT sessions as the JIT'd code no longer changes, only the 318absolute symbol definition does. 319 320For process and library symbols the DynamicLibrarySearchGenerator utility (See 321:ref:`How to Add Process and Library Symbols to JITDylibs 322<ProcessAndLibrarySymbols>`) can be used to automatically build absolute 323symbol mappings for you. However the absoluteSymbols function is still useful 324for making non-global objects in your JIT visible to JIT'd code. For example, 325imagine that your JIT standard library needs access to your JIT object to make 326some calls. We could bake the address of your object into the library, but then 327it would need to be recompiled for each session: 328 329.. code-block: c++ 330 331 // From standard library for JIT'd code: 332 333 class MyJIT { 334 public: 335 void log(const char *Msg); 336 }; 337 338 void log(const char *Msg) { ((MyJIT*)0x1234)->log(Msg); } 339 340We can turn this into a symbolic reference in the JIT standard library: 341 342.. code-block: c++ 343 344 extern MyJIT *__MyJITInstance; 345 346 void log(const char *Msg) { __MyJITInstance->log(Msg); } 347 348And then make our JIT object visible to the JIT standard library with an 349absolute symbol definition when the JIT is started: 350 351.. code-block: c++ 352 353 MyJIT J = ...; 354 355 auto &JITStdLibJD = ... ; 356 357 JITStdLibJD.define(absoluteSymbols(SymbolMap({ 358 { Mangle("__MyJITInstance"), 359 { pointerToJITTargetAddress(&J), JITSymbolFlags() } } 360 }); 361 362Aliases and Reexports 363--------------------- 364 365Aliases and reexports allow you to define new symbols that map to existing 366symbols. This can be useful for changing linkage relationships between symbols 367across sessions without having to recompile code. For example, imagine that 368JIT'd code has access to a log function, ``void log(const char*)`` for which 369there are two implementations in the JIT standard library: ``log_fast`` and 370``log_detailed``. Your JIT can choose which one of these definitions will be 371used when the ``log`` symbol is referenced by setting up an alias at JIT startup 372time: 373 374.. code-block: c++ 375 376 auto &JITStdLibJD = ... ; 377 378 auto LogImplementationSymbol = 379 Verbose ? Mangle("log_detailed") : Mangle("log_fast"); 380 381 JITStdLibJD.define( 382 symbolAliases(SymbolAliasMap({ 383 { Mangle("log"), 384 { LogImplementationSymbol 385 JITSymbolFlags::Exported | JITSymbolFlags::Callable } } 386 }); 387 388The ``symbolAliases`` function allows you to define aliases within a single 389JITDylib. The ``reexports`` function provides the same functionality, but 390operates across JITDylib boundaries. E.g. 391 392.. code-block: c++ 393 394 auto &JD1 = ... ; 395 auto &JD2 = ... ; 396 397 // Make 'bar' in JD2 an alias for 'foo' from JD1. 398 JD2.define( 399 reexports(JD1, SymbolAliasMap({ 400 { Mangle("bar"), { Mangle("foo"), JITSymbolFlags::Exported } } 401 }); 402 403The reexports utility can be handy for composing a single JITDylib interface by 404re-exporting symbols from several other JITDylibs. 405 406.. _Laziness: 407 408Laziness 409======== 410 411Laziness in ORC is provided by a utility called "lazy reexports". A lazy 412reexport is similar to a regular reexport or alias: It provides a new name for 413an existing symbol. Unlike regular reexports however, lookups of lazy reexports 414do not trigger immediate materialization of the reexported symbol. Instead, they 415only trigger materialization of a function stub. This function stub is 416initialized to point at a *lazy call-through*, which provides reentry into the 417JIT. If the stub is called at runtime then the lazy call-through will look up 418the reexported symbol (triggering materialization for it if necessary), update 419the stub (to call directly to the reexported symbol on subsequent calls), and 420then return via the reexported symbol. By re-using the existing symbol lookup 421mechanism, lazy reexports inherit the same concurrency guarantees: calls to lazy 422reexports can be made from multiple threads concurrently, and the reexported 423symbol can be any state of compilation (uncompiled, already in the process of 424being compiled, or already compiled) and the call will succeed. This allows 425laziness to be safely mixed with features like remote compilation, concurrent 426compilation, concurrent JIT'd code, and speculative compilation. 427 428There is one other key difference between regular reexports and lazy reexports 429that some clients must be aware of: The address of a lazy reexport will be 430*different* from the address of the reexported symbol (whereas a regular 431reexport is guaranteed to have the same address as the reexported symbol). 432Clients who care about pointer equality will generally want to use the address 433of the reexport as the canonical address of the reexported symbol. This will 434allow the address to be taken without forcing materialization of the reexport. 435 436Usage example: 437 438If JITDylib ``JD`` contains definitions for symbols ``foo_body`` and 439``bar_body``, we can create lazy entry points ``Foo`` and ``Bar`` in JITDylib 440``JD2`` by calling: 441 442.. code-block:: c++ 443 444 auto ReexportFlags = JITSymbolFlags::Exported | JITSymbolFlags::Callable; 445 JD2.define( 446 lazyReexports(CallThroughMgr, StubsMgr, JD, 447 SymbolAliasMap({ 448 { Mangle("foo"), { Mangle("foo_body"), ReexportedFlags } }, 449 { Mangle("bar"), { Mangle("bar_body"), ReexportedFlags } } 450 })); 451 452A full example of how to use lazyReexports with the LLJIT class can be found at 453``llvm_project/llvm/examples/LLJITExamples/LLJITWithLazyReexports``. 454 455Supporting Custom Compilers 456=========================== 457 458TBD. 459 460Transitioning from ORCv1 to ORCv2 461================================= 462 463Since LLVM 7.0, new ORC development work has focused on adding support for 464concurrent JIT compilation. The new APIs (including new layer interfaces and 465implementations, and new utilities) that support concurrency are collectively 466referred to as ORCv2, and the original, non-concurrent layers and utilities 467are now referred to as ORCv1. 468 469The majority of the ORCv1 layers and utilities were renamed with a 'Legacy' 470prefix in LLVM 8.0, and have deprecation warnings attached in LLVM 9.0. In LLVM 47110.0 ORCv1 will be removed entirely. 472 473Transitioning from ORCv1 to ORCv2 should be easy for most clients. Most of the 474ORCv1 layers and utilities have ORCv2 counterparts [2]_ that can be directly 475substituted. However there are some design differences between ORCv1 and ORCv2 476to be aware of: 477 478 1. ORCv2 fully adopts the JIT-as-linker model that began with MCJIT. Modules 479 (and other program representations, e.g. Object Files) are no longer added 480 directly to JIT classes or layers. Instead, they are added to ``JITDylib`` 481 instances *by* layers. The ``JITDylib`` determines *where* the definitions 482 reside, the layers determine *how* the definitions will be compiled. 483 Linkage relationships between ``JITDylibs`` determine how inter-module 484 references are resolved, and symbol resolvers are no longer used. See the 485 section `Design Overview`_ for more details. 486 487 Unless multiple JITDylibs are needed to model linkage relationships, ORCv1 488 clients should place all code in a single JITDylib. 489 MCJIT clients should use LLJIT (see `LLJIT and LLLazyJIT`_), and can place 490 code in LLJIT's default created main JITDylib (See 491 ``LLJIT::getMainJITDylib()``). 492 493 2. All JIT stacks now need an ``ExecutionSession`` instance. ExecutionSession 494 manages the string pool, error reporting, synchronization, and symbol 495 lookup. 496 497 3. ORCv2 uses uniqued strings (``SymbolStringPtr`` instances) rather than 498 string values in order to reduce memory overhead and improve lookup 499 performance. See the subsection `How to manage symbol strings`_. 500 501 4. IR layers require ThreadSafeModule instances, rather than 502 std::unique_ptr<Module>s. ThreadSafeModule is a wrapper that ensures that 503 Modules that use the same LLVMContext are not accessed concurrently. 504 See `How to use ThreadSafeModule and ThreadSafeContext`_. 505 506 5. Symbol lookup is no longer handled by layers. Instead, there is a 507 ``lookup`` method on JITDylib that takes a list of JITDylibs to scan. 508 509 .. code-block:: c++ 510 511 ExecutionSession ES; 512 JITDylib &JD1 = ...; 513 JITDylib &JD2 = ...; 514 515 auto Sym = ES.lookup({&JD1, &JD2}, ES.intern("_main")); 516 517 6. Module removal is not yet supported. There is no equivalent of the 518 layer concept removeModule/removeObject methods. Work on resource tracking 519 and removal in ORCv2 is ongoing. 520 521For code examples and suggestions of how to use the ORCv2 APIs, please see 522the section `How-tos`_. 523 524How-tos 525======= 526 527How to manage symbol strings 528---------------------------- 529 530Symbol strings in ORC are uniqued to improve lookup performance, reduce memory 531overhead, and allow symbol names to function as efficient keys. To get the 532unique ``SymbolStringPtr`` for a string value, call the 533``ExecutionSession::intern`` method: 534 535 .. code-block:: c++ 536 537 ExecutionSession ES; 538 /// ... 539 auto MainSymbolName = ES.intern("main"); 540 541If you wish to perform lookup using the C/IR name of a symbol you will also 542need to apply the platform linker-mangling before interning the string. On 543Linux this mangling is a no-op, but on other platforms it usually involves 544adding a prefix to the string (e.g. '_' on Darwin). The mangling scheme is 545based on the DataLayout for the target. Given a DataLayout and an 546ExecutionSession, you can create a MangleAndInterner function object that 547will perform both jobs for you: 548 549 .. code-block:: c++ 550 551 ExecutionSession ES; 552 const DataLayout &DL = ...; 553 MangleAndInterner Mangle(ES, DL); 554 555 // ... 556 557 // Portable IR-symbol-name lookup: 558 auto Sym = ES.lookup({&MainJD}, Mangle("main")); 559 560How to create JITDylibs and set up linkage relationships 561-------------------------------------------------------- 562 563In ORC, all symbol definitions reside in JITDylibs. JITDylibs are created by 564calling the ``ExecutionSession::createJITDylib`` method with a unique name: 565 566 .. code-block:: c++ 567 568 ExecutionSession ES; 569 auto &JD = ES.createJITDylib("libFoo.dylib"); 570 571The JITDylib is owned by the ``ExecutionEngine`` instance and will be freed 572when it is destroyed. 573 574How to use ThreadSafeModule and ThreadSafeContext 575------------------------------------------------- 576 577ThreadSafeModule and ThreadSafeContext are wrappers around Modules and 578LLVMContexts respectively. A ThreadSafeModule is a pair of a 579std::unique_ptr<Module> and a (possibly shared) ThreadSafeContext value. A 580ThreadSafeContext is a pair of a std::unique_ptr<LLVMContext> and a lock. 581This design serves two purposes: providing a locking scheme and lifetime 582management for LLVMContexts. The ThreadSafeContext may be locked to prevent 583accidental concurrent access by two Modules that use the same LLVMContext. 584The underlying LLVMContext is freed once all ThreadSafeContext values pointing 585to it are destroyed, allowing the context memory to be reclaimed as soon as 586the Modules referring to it are destroyed. 587 588ThreadSafeContexts can be explicitly constructed from a 589std::unique_ptr<LLVMContext>: 590 591 .. code-block:: c++ 592 593 ThreadSafeContext TSCtx(std::make_unique<LLVMContext>()); 594 595ThreadSafeModules can be constructed from a pair of a std::unique_ptr<Module> 596and a ThreadSafeContext value. ThreadSafeContext values may be shared between 597multiple ThreadSafeModules: 598 599 .. code-block:: c++ 600 601 ThreadSafeModule TSM1( 602 std::make_unique<Module>("M1", *TSCtx.getContext()), TSCtx); 603 604 ThreadSafeModule TSM2( 605 std::make_unique<Module>("M2", *TSCtx.getContext()), TSCtx); 606 607Before using a ThreadSafeContext, clients should ensure that either the context 608is only accessible on the current thread, or that the context is locked. In the 609example above (where the context is never locked) we rely on the fact that both 610``TSM1`` and ``TSM2``, and TSCtx are all created on one thread. If a context is 611going to be shared between threads then it must be locked before any accessing 612or creating any Modules attached to it. E.g. 613 614 .. code-block:: c++ 615 616 ThreadSafeContext TSCtx(std::make_unique<LLVMContext>()); 617 618 ThreadPool TP(NumThreads); 619 JITStack J; 620 621 for (auto &ModulePath : ModulePaths) { 622 TP.async( 623 [&]() { 624 auto Lock = TSCtx.getLock(); 625 auto M = loadModuleOnContext(ModulePath, TSCtx.getContext()); 626 J.addModule(ThreadSafeModule(std::move(M), TSCtx)); 627 }); 628 } 629 630 TP.wait(); 631 632To make exclusive access to Modules easier to manage the ThreadSafeModule class 633provides a convenience function, ``withModuleDo``, that implicitly (1) locks the 634associated context, (2) runs a given function object, (3) unlocks the context, 635and (3) returns the result generated by the function object. E.g. 636 637 .. code-block:: c++ 638 639 ThreadSafeModule TSM = getModule(...); 640 641 // Dump the module: 642 size_t NumFunctionsInModule = 643 TSM.withModuleDo( 644 [](Module &M) { // <- Context locked before entering lambda. 645 return M.size(); 646 } // <- Context unlocked after leaving. 647 ); 648 649Clients wishing to maximize possibilities for concurrent compilation will want 650to create every new ThreadSafeModule on a new ThreadSafeContext. For this 651reason a convenience constructor for ThreadSafeModule is provided that implicitly 652constructs a new ThreadSafeContext value from a std::unique_ptr<LLVMContext>: 653 654 .. code-block:: c++ 655 656 // Maximize concurrency opportunities by loading every module on a 657 // separate context. 658 for (const auto &IRPath : IRPaths) { 659 auto Ctx = std::make_unique<LLVMContext>(); 660 auto M = std::make_unique<LLVMContext>("M", *Ctx); 661 CompileLayer.add(MainJD, ThreadSafeModule(std::move(M), std::move(Ctx))); 662 } 663 664Clients who plan to run single-threaded may choose to save memory by loading 665all modules on the same context: 666 667 .. code-block:: c++ 668 669 // Save memory by using one context for all Modules: 670 ThreadSafeContext TSCtx(std::make_unique<LLVMContext>()); 671 for (const auto &IRPath : IRPaths) { 672 ThreadSafeModule TSM(parsePath(IRPath, *TSCtx.getContext()), TSCtx); 673 CompileLayer.add(MainJD, ThreadSafeModule(std::move(TSM)); 674 } 675 676.. _ProcessAndLibrarySymbols: 677 678How to Add Process and Library Symbols to the JITDylibs 679======================================================= 680 681JIT'd code typically needs access to symbols in the host program or in 682supporting libraries. References to process symbols can be "baked in" to code 683as it is compiled by turning external references into pre-resolved integer 684constants, however this ties the JIT'd code to the current process's virtual 685memory layout (meaning that it can not be cached between runs) and makes 686debugging lower level program representations difficult (as all external 687references are opaque integer values). A bettor solution is to maintain symbolic 688external references and let the jit-linker bind them for you at runtime. To 689allow the JIT linker to find these external definitions their addresses must 690be added to a JITDylib that the JIT'd definitions link against. 691 692Adding definitions for external symbols could be done using the absoluteSymbols 693function: 694 695 .. code-block:: c++ 696 697 const DataLayout &DL = getDataLayout(); 698 MangleAndInterner Mangle(ES, DL); 699 700 auto &JD = ES.createJITDylib("main"); 701 702 JD.define( 703 absoluteSymbols({ 704 { Mangle("puts"), pointerToJITTargetAddress(&puts)}, 705 { Mangle("gets"), pointerToJITTargetAddress(&getS)} 706 })); 707 708Manually adding absolute symbols for a large or changing interface is cumbersome 709however, so ORC provides an alternative to generate new definitions on demand: 710*definition generators*. If a definition generator is attached to a JITDylib, 711then any unsuccessful lookup on that JITDylib will fall back to calling the 712definition generator, and the definition generator may choose to generate a new 713definition for the missing symbols. Of particular use here is the 714``DynamicLibrarySearchGenerator`` utility. This can be used to reflect the whole 715exported symbol set of the process or a specific dynamic library, or a subset 716of either of these determined by a predicate. 717 718For example, to load the whole interface of a runtime library: 719 720 .. code-block:: c++ 721 722 const DataLayout &DL = getDataLayout(); 723 auto &JD = ES.createJITDylib("main"); 724 725 JD.setGenerator(DynamicLibrarySearchGenerator::Load("/path/to/lib" 726 DL.getGlobalPrefix())); 727 728 // IR added to JD can now link against all symbols exported by the library 729 // at '/path/to/lib'. 730 CompileLayer.add(JD, loadModule(...)); 731 732Or, to expose an allowed set of symbols from the main process: 733 734 .. code-block:: c++ 735 736 const DataLayout &DL = getDataLayout(); 737 MangleAndInterner Mangle(ES, DL); 738 739 auto &JD = ES.createJITDylib("main"); 740 741 DenseSet<SymbolStringPtr> AllowList({ 742 Mangle("puts"), 743 Mangle("gets") 744 }); 745 746 // Use GetForCurrentProcess with a predicate function that checks the 747 // allowed list. 748 JD.setGenerator( 749 DynamicLibrarySearchGenerator::GetForCurrentProcess( 750 DL.getGlobalPrefix(), 751 [&](const SymbolStringPtr &S) { return AllowList.count(S); })); 752 753 // IR added to JD can now link against any symbols exported by the process 754 // and contained in the list. 755 CompileLayer.add(JD, loadModule(...)); 756 757Future Features 758=============== 759 760TBD: Speculative compilation. Object Caches. 761 762.. [1] Formats/architectures vary in terms of supported features. MachO and 763 ELF tend to have better support than COFF. Patches very welcome! 764 765.. [2] The ``LazyEmittingLayer``, ``RemoteObjectClientLayer`` and 766 ``RemoteObjectServerLayer`` do not have counterparts in the new 767 system. In the case of ``LazyEmittingLayer`` it was simply no longer 768 needed: in ORCv2, deferring compilation until symbols are looked up is 769 the default. The removal of ``RemoteObjectClientLayer`` and 770 ``RemoteObjectServerLayer`` means that JIT stacks can no longer be split 771 across processes, however this functionality appears not to have been 772 used. 773 774.. [3] Weak definitions are currently handled correctly within dylibs, but if 775 multiple dylibs provide a weak definition of a symbol then each will end 776 up with its own definition (similar to how weak definitions are handled 777 in Windows DLLs). This will be fixed in the future. 778