1=============================
2Advanced Build Configurations
3=============================
4
5.. contents::
6   :local:
7
8Introduction
9============
10
11`CMake <http://www.cmake.org/>`_ is a cross-platform build-generator tool. CMake
12does not build the project, it generates the files needed by your build tool
13(GNU make, Visual Studio, etc.) for building LLVM.
14
15If **you are a new contributor**, please start with the :doc:`GettingStarted` or
16:doc:`CMake` pages. This page is intended for users doing more complex builds.
17
18Many of the examples below are written assuming specific CMake Generators.
19Unless otherwise explicitly called out these commands should work with any CMake
20generator.
21
22Bootstrap Builds
23================
24
25The Clang CMake build system supports bootstrap (aka multi-stage) builds. At a
26high level a multi-stage build is a chain of builds that pass data from one
27stage into the next. The most common and simple version of this is a traditional
28bootstrap build.
29
30In a simple two-stage bootstrap build, we build clang using the system compiler,
31then use that just-built clang to build clang again. In CMake this simplest form
32of a bootstrap build can be configured with a single option,
33CLANG_ENABLE_BOOTSTRAP.
34
35.. code-block:: console
36
37  $ cmake -G Ninja -DCLANG_ENABLE_BOOTSTRAP=On <path to source>
38  $ ninja stage2
39
40This command itself isn't terribly useful because it assumes default
41configurations for each stage. The next series of examples utilize CMake cache
42scripts to provide more complex options.
43
44By default, only a few CMake options will be passed between stages.
45The list, called _BOOTSTRAP_DEFAULT_PASSTHROUGH, is defined in clang/CMakeLists.txt.
46To force the passing of the variables between stages, use the -DCLANG_BOOTSTRAP_PASSTHROUGH
47CMake option, each variable separated by a ";". As example:
48
49.. code-block:: console
50
51  $ cmake -G Ninja -DCLANG_ENABLE_BOOTSTRAP=On -DCLANG_BOOTSTRAP_PASSTHROUGH="CMAKE_INSTALL_PREFIX;CMAKE_VERBOSE_MAKEFILE" <path to source>
52  $ ninja stage2
53
54CMake options starting by ``BOOTSTRAP_`` will be passed only to the stage2 build.
55This gives the opportunity to use Clang specific build flags.
56For example, the following CMake call will enabled '-fno-addrsig' only during
57the stage2 build for C and C++.
58
59.. code-block:: console
60
61  $ cmake [..]  -DBOOTSTRAP_CMAKE_CXX_FLAGS='-fno-addrsig' -DBOOTSTRAP_CMAKE_C_FLAGS='-fno-addrsig' [..]
62
63The clang build system refers to builds as stages. A stage1 build is a standard
64build using the compiler installed on the host, and a stage2 build is built
65using the stage1 compiler. This nomenclature holds up to more stages too. In
66general a stage*n* build is built using the output from stage*n-1*.
67
68Apple Clang Builds (A More Complex Bootstrap)
69=============================================
70
71Apple's Clang builds are a slightly more complicated example of the simple
72bootstrapping scenario. Apple Clang is built using a 2-stage build.
73
74The stage1 compiler is a host-only compiler with some options set. The stage1
75compiler is a balance of optimization vs build time because it is a throwaway.
76The stage2 compiler is the fully optimized compiler intended to ship to users.
77
78Setting up these compilers requires a lot of options. To simplify the
79configuration the Apple Clang build settings are contained in CMake Cache files.
80You can build an Apple Clang compiler using the following commands:
81
82.. code-block:: console
83
84  $ cmake -G Ninja -C <path to clang>/cmake/caches/Apple-stage1.cmake <path to source>
85  $ ninja stage2-distribution
86
87This CMake invocation configures the stage1 host compiler, and sets
88CLANG_BOOTSTRAP_CMAKE_ARGS to pass the Apple-stage2.cmake cache script to the
89stage2 configuration step.
90
91When you build the stage2-distribution target it builds the minimal stage1
92compiler and required tools, then configures and builds the stage2 compiler
93based on the settings in Apple-stage2.cmake.
94
95This pattern of using cache scripts to set complex settings, and specifically to
96make later stage builds include cache scripts is common in our more advanced
97build configurations.
98
99Multi-stage PGO
100===============
101
102Profile-Guided Optimizations (PGO) is a really great way to optimize the code
103clang generates. Our multi-stage PGO builds are a workflow for generating PGO
104profiles that can be used to optimize clang.
105
106At a high level, the way PGO works is that you build an instrumented compiler,
107then you run the instrumented compiler against sample source files. While the
108instrumented compiler runs it will output a bunch of files containing
109performance counters (.profraw files). After generating all the profraw files
110you use llvm-profdata to merge the files into a single profdata file that you
111can feed into the LLVM_PROFDATA_FILE option.
112
113Our PGO.cmake cache script automates that whole process. You can use it by
114running:
115
116.. code-block:: console
117
118  $ cmake -G Ninja -C <path_to_clang>/cmake/caches/PGO.cmake <source dir>
119  $ ninja stage2-instrumented-generate-profdata
120
121If you let that run for a few hours or so, it will place a profdata file in your
122build directory. This takes a really long time because it builds clang twice,
123and you *must* have compiler-rt in your build tree.
124
125This process uses any source files under the perf-training directory as training
126data as long as the source files are marked up with LIT-style RUN lines.
127
128After it finishes you can use “find . -name clang.profdata” to find it, but it
129should be at a path something like:
130
131.. code-block:: console
132
133  <build dir>/tools/clang/stage2-instrumented-bins/utils/perf-training/clang.profdata
134
135You can feed that file into the LLVM_PROFDATA_FILE option when you build your
136optimized compiler.
137
138The PGO came cache has a slightly different stage naming scheme than other
139multi-stage builds. It generates three stages; stage1, stage2-instrumented, and
140stage2. Both of the stage2 builds are built using the stage1 compiler.
141
142The PGO came cache generates the following additional targets:
143
144**stage2-instrumented**
145  Builds a stage1 x86 compiler, runtime, and required tools (llvm-config,
146  llvm-profdata) then uses that compiler to build an instrumented stage2 compiler.
147
148**stage2-instrumented-generate-profdata**
149  Depends on "stage2-instrumented" and will use the instrumented compiler to
150  generate profdata based on the training files in <clang>/utils/perf-training
151
152**stage2**
153  Depends of "stage2-instrumented-generate-profdata" and will use the stage1
154  compiler with the stage2 profdata to build a PGO-optimized compiler.
155
156**stage2-check-llvm**
157  Depends on stage2 and runs check-llvm using the stage2 compiler.
158
159**stage2-check-clang**
160  Depends on stage2 and runs check-clang using the stage2 compiler.
161
162**stage2-check-all**
163  Depends on stage2 and runs check-all using the stage2 compiler.
164
165**stage2-test-suite**
166  Depends on stage2 and runs the test-suite using the stage3 compiler (requires
167  in-tree test-suite).
168
1693-Stage Non-Determinism
170=======================
171
172In the ancient lore of compilers non-determinism is like the multi-headed hydra.
173Whenever its head pops up, terror and chaos ensue.
174
175Historically one of the tests to verify that a compiler was deterministic would
176be a three stage build. The idea of a three stage build is you take your sources
177and build a compiler (stage1), then use that compiler to rebuild the sources
178(stage2), then you use that compiler to rebuild the sources a third time
179(stage3) with an identical configuration to the stage2 build. At the end of
180this, you have a stage2 and stage3 compiler that should be bit-for-bit
181identical.
182
183You can perform one of these 3-stage builds with LLVM & clang using the
184following commands:
185
186.. code-block:: console
187
188  $ cmake -G Ninja -C <path_to_clang>/cmake/caches/3-stage.cmake <source dir>
189  $ cmake --build . --target stage3 --parallel
190
191After the build you can compare the stage2 & stage3 compilers.
192