1<!-- doc/src/sgml/jit.sgml -->
2
3<chapter id="jit">
4 <title>Just-in-Time Compilation (<acronym>JIT</acronym>)</title>
5
6 <indexterm zone="jit">
7  <primary><acronym>JIT</acronym></primary>
8 </indexterm>
9
10 <indexterm>
11  <primary>Just-In-Time compilation</primary>
12  <see><acronym>JIT</acronym></see>
13 </indexterm>
14
15 <para>
16  This chapter explains what just-in-time compilation is, and how it can be
17  configured in <productname>PostgreSQL</productname>.
18 </para>
19
20 <sect1 id="jit-reason">
21  <title>What Is <acronym>JIT</acronym> compilation?</title>
22
23  <para>
24   Just-in-Time (<acronym>JIT</acronym>) compilation is the process of turning
25   some form of interpreted program evaluation into a native program, and
26   doing so at run time.
27   For example, instead of using general-purpose code that can evaluate
28   arbitrary SQL expressions to evaluate a particular SQL predicate
29   like <literal>WHERE a.col = 3</literal>, it is possible to generate a
30   function that is specific to that expression and can be natively executed
31   by the CPU, yielding a speedup.
32  </para>
33
34  <para>
35   <productname>PostgreSQL</productname> has builtin support to perform
36   <acronym>JIT</acronym> compilation using <ulink
37   url="https://llvm.org/"><productname>LLVM</productname></ulink> when
38   <productname>PostgreSQL</productname> is built with
39   <link linkend="configure-with-llvm"><literal>--with-llvm</literal></link>.
40  </para>
41
42  <para>
43   See <filename>src/backend/jit/README</filename> for further details.
44  </para>
45
46  <sect2 id="jit-accelerated-operations">
47   <title><acronym>JIT</acronym> Accelerated Operations</title>
48   <para>
49    Currently <productname>PostgreSQL</productname>'s <acronym>JIT</acronym>
50    implementation has support for accelerating expression evaluation and
51    tuple deforming.  Several other operations could be accelerated in the
52    future.
53   </para>
54   <para>
55    Expression evaluation is used to evaluate <literal>WHERE</literal>
56    clauses, target lists, aggregates and projections. It can be accelerated
57    by generating code specific to each case.
58   </para>
59   <para>
60    Tuple deforming is the process of transforming an on-disk tuple (see <xref
61    linkend="storage-tuple-layout"/>) into its in-memory representation.
62    It can be accelerated by creating a function specific to the table layout
63    and the number of columns to be extracted.
64   </para>
65  </sect2>
66
67  <sect2 id="jit-inlining">
68   <title>Inlining</title>
69   <para>
70    <productname>PostgreSQL</productname> is very extensible and allows new
71    data types, functions, operators and other database objects to be defined;
72    see <xref linkend="extend"/>. In fact the built-in objects are implemented
73    using nearly the same mechanisms.  This extensibility implies some
74    overhead, for example due to function calls (see <xref linkend="xfunc"/>).
75    To reduce that overhead, <acronym>JIT</acronym> compilation can inline the
76    bodies of small functions into the expressions using them. That allows a
77    significant percentage of the overhead to be optimized away.
78   </para>
79  </sect2>
80
81  <sect2 id="jit-optimization">
82   <title>Optimization</title>
83   <para>
84    <productname>LLVM</productname> has support for optimizing generated
85    code. Some of the optimizations are cheap enough to be performed whenever
86    <acronym>JIT</acronym> is used, while others are only beneficial for
87    longer-running queries.
88    See <ulink url="https://llvm.org/docs/Passes.html#transform-passes"/> for
89    more details about optimizations.
90   </para>
91  </sect2>
92
93 </sect1>
94
95 <sect1 id="jit-decision">
96  <title>When to <acronym>JIT</acronym>?</title>
97
98  <para>
99   <acronym>JIT</acronym> compilation is beneficial primarily for long-running
100   CPU-bound queries. Frequently these will be analytical queries.  For short
101   queries the added overhead of performing <acronym>JIT</acronym> compilation
102   will often be higher than the time it can save.
103  </para>
104
105  <para>
106   To determine whether <acronym>JIT</acronym> compilation should be used,
107   the total estimated cost of a query (see
108   <xref linkend="planner-stats-details"/> and
109   <xref linkend="runtime-config-query-constants"/>) is used.
110   The estimated cost of the query will be compared with the setting of <xref
111   linkend="guc-jit-above-cost"/>. If the cost is higher,
112   <acronym>JIT</acronym> compilation will be performed.
113   Two further decisions are then needed.
114   Firstly, if the estimated cost is more
115   than the setting of <xref linkend="guc-jit-inline-above-cost"/>, short
116   functions and operators used in the query will be inlined.
117   Secondly, if the estimated cost is more than the setting of <xref
118   linkend="guc-jit-optimize-above-cost"/>, expensive optimizations are
119   applied to improve the generated code.
120   Each of these options increases the <acronym>JIT</acronym> compilation
121   overhead, but can reduce query execution time considerably.
122  </para>
123
124  <para>
125   These cost-based decisions will be made at plan time, not execution
126   time. This means that when prepared statements are in use, and a generic
127   plan is used (see <xref linkend="sql-prepare"/>), the values of the
128   configuration parameters in effect at prepare time control the decisions,
129   not the settings at execution time.
130  </para>
131
132  <note>
133   <para>
134    If <xref linkend="guc-jit"/> is set to <literal>off</literal>, or if no
135    <acronym>JIT</acronym> implementation is available (for example because
136    the server was compiled without <literal>--with-llvm</literal>),
137    <acronym>JIT</acronym> will not be performed, even if it would be
138    beneficial based on the above criteria.  Setting <xref linkend="guc-jit"/>
139    to <literal>off</literal> has effects at both plan and execution time.
140   </para>
141  </note>
142
143  <para>
144   <xref linkend="sql-explain"/> can be used to see whether
145   <acronym>JIT</acronym> is used or not.  As an example, here is a query that
146   is not using <acronym>JIT</acronym>:
147<screen>
148=# EXPLAIN ANALYZE SELECT SUM(relpages) FROM pg_class;
149                                                 QUERY PLAN
150-------------------------------------------------------------------------------------------------------------
151 Aggregate  (cost=16.27..16.29 rows=1 width=8) (actual time=0.303..0.303 rows=1 loops=1)
152   ->  Seq Scan on pg_class  (cost=0.00..15.42 rows=342 width=4) (actual time=0.017..0.111 rows=356 loops=1)
153 Planning Time: 0.116 ms
154 Execution Time: 0.365 ms
155(4 rows)
156</screen>
157   Given the cost of the plan, it is entirely reasonable that no
158   <acronym>JIT</acronym> was used; the cost of <acronym>JIT</acronym> would
159   have been bigger than the potential savings. Adjusting the cost limits
160   will lead to <acronym>JIT</acronym> use:
161<screen>
162=# SET jit_above_cost = 10;
163SET
164=# EXPLAIN ANALYZE SELECT SUM(relpages) FROM pg_class;
165                                                 QUERY PLAN
166-------------------------------------------------------------------------------------------------------------
167 Aggregate  (cost=16.27..16.29 rows=1 width=8) (actual time=6.049..6.049 rows=1 loops=1)
168   ->  Seq Scan on pg_class  (cost=0.00..15.42 rows=342 width=4) (actual time=0.019..0.052 rows=356 loops=1)
169 Planning Time: 0.133 ms
170 JIT:
171   Functions: 3
172   Options: Inlining false, Optimization false, Expressions true, Deforming true
173   Timing: Generation 1.259 ms, Inlining 0.000 ms, Optimization 0.797 ms, Emission 5.048 ms, Total 7.104 ms
174 Execution Time: 7.416 ms
175</screen>
176   As visible here, <acronym>JIT</acronym> was used, but inlining and
177   expensive optimization were not. If <xref
178   linkend="guc-jit-inline-above-cost"/> or <xref
179   linkend="guc-jit-optimize-above-cost"/> were also lowered,
180   that would change.
181  </para>
182 </sect1>
183
184 <sect1 id="jit-configuration" xreflabel="JIT Configuration">
185  <title>Configuration</title>
186
187  <para>
188   The configuration variable
189   <xref linkend="guc-jit"/> determines whether <acronym>JIT</acronym>
190   compilation is enabled or disabled.
191   If it is enabled, the configuration variables
192   <xref linkend="guc-jit-above-cost"/>, <xref
193   linkend="guc-jit-inline-above-cost"/>, and <xref
194   linkend="guc-jit-optimize-above-cost"/> determine
195   whether <acronym>JIT</acronym> compilation is performed for a query,
196   and how much effort is spent doing so.
197  </para>
198
199  <para>
200   <xref linkend="guc-jit-provider"/> determines which <acronym>JIT</acronym>
201   implementation is used. It is rarely required to be changed. See <xref
202   linkend="jit-pluggable"/>.
203  </para>
204
205  <para>
206   For development and debugging purposes a few additional configuration
207   parameters exist, as described in
208   <xref linkend="runtime-config-developer"/>.
209  </para>
210 </sect1>
211
212 <sect1 id="jit-extensibility">
213  <title>Extensibility</title>
214
215  <sect2 id="jit-extensibility-bitcode">
216   <title>Inlining Support for Extensions</title>
217   <para>
218    <productname>PostgreSQL</productname>'s <acronym>JIT</acronym>
219    implementation can inline the bodies of functions
220    of types <literal>C</literal> and <literal>internal</literal>, as well as
221    operators based on such functions.  To do so for functions in extensions,
222    the definitions of those functions need to be made available.
223    When using <link linkend="extend-pgxs">PGXS</link> to build an extension
224    against a server that has been compiled with LLVM JIT support, the
225    relevant files will be built and installed automatically.
226   </para>
227
228   <para>
229    The relevant files have to be installed into
230    <filename>$pkglibdir/bitcode/$extension/</filename> and a summary of them
231    into <filename>$pkglibdir/bitcode/$extension.index.bc</filename>, where
232    <literal>$pkglibdir</literal> is the directory returned by
233    <literal>pg_config --pkglibdir</literal> and <literal>$extension</literal>
234    is the base name of the extension's shared library.
235
236    <note>
237     <para>
238      For functions built into <productname>PostgreSQL</productname> itself,
239      the bitcode is installed into
240      <literal>$pkglibdir/bitcode/postgres</literal>.
241     </para>
242    </note>
243   </para>
244  </sect2>
245
246  <sect2 id="jit-pluggable">
247   <title>Pluggable <acronym>JIT</acronym> Providers</title>
248
249   <para>
250    <productname>PostgreSQL</productname> provides a <acronym>JIT</acronym>
251    implementation based on <productname>LLVM</productname>.  The interface to
252    the <acronym>JIT</acronym> provider is pluggable and the provider can be
253    changed without recompiling (although currently, the build process only
254    provides inlining support data for <productname>LLVM</productname>).
255    The active provider is chosen via the setting
256    <xref linkend="guc-jit-provider"/>.
257   </para>
258
259   <sect3>
260    <title><acronym>JIT</acronym> Provider Interface</title>
261    <para>
262     A <acronym>JIT</acronym> provider is loaded by dynamically loading the
263     named shared library. The normal library search path is used to locate
264     the library. To provide the required <acronym>JIT</acronym> provider
265     callbacks and to indicate that the library is actually a
266     <acronym>JIT</acronym> provider, it needs to provide a C function named
267     <function>_PG_jit_provider_init</function>. This function is passed a
268     struct that needs to be filled with the callback function pointers for
269     individual actions:
270<programlisting>
271struct JitProviderCallbacks
272{
273    JitProviderResetAfterErrorCB reset_after_error;
274    JitProviderReleaseContextCB release_context;
275    JitProviderCompileExprCB compile_expr;
276};
277
278extern void _PG_jit_provider_init(JitProviderCallbacks *cb);
279</programlisting>
280    </para>
281   </sect3>
282  </sect2>
283 </sect1>
284
285</chapter>
286