1.. SPDX-License-Identifier: GPL-3.0-or-later
2
3.. _modules-api:
4
5*********************
6Modules API reference
7*********************
8
9.. contents::
10 :depth: 1
11 :local:
12
13Supported languages
14===================
15
16Currently modules written in C and Lua(JIT) are supported.
17
18The anatomy of an extension
19===========================
20
21A module is a shared object or script defining specific functions/fields; here's an overview.
22
23.. csv-table::
24 :header: "C", "Lua", "Params", "Comment"
25
26 "``X_api()`` [#]_", "", "", "API version"
27 "``X_init()``", "``X.init()``", "``module``", "Constructor"
28 "``X_deinit()``", "``X.deinit()``", "``module``", "Destructor"
29 "``X_config()``", "``X.config()``", "``module, str``", "Configuration"
30 "``X_layer``", "``X.layer``", "", ":ref:`Module layer <lib-layers>`"
31 "``X_props``", "", "", "List of properties"
32
33.. [#] Mandatory symbol; defined by using :c:func:`KR_MODULE_EXPORT`.
34
35The ``X`` corresponds to the module name; if the module name is ``hints``, the prefix for constructor would be ``hints_init()``.
36More details are in docs for the :c:type:`kr_module` and :c:type:`kr_layer_api` structures.
37
38.. note::
39 The modules get ordered -- by default in the same as the order in which they were loaded. The loading command can specify where in the order the module should be positioned.
40
41
42Writing a module in Lua
43=======================
44
45The probably most convenient way of writing modules is Lua since you can use already installed modules
46from system and have first-class access to the scripting engine. You can also tap to all the events, that
47the C API has access to, but keep in mind that transitioning from the C to Lua function is slower than
48the other way round, especially when JIT-compilation is taken into account.
49
50.. note:: The Lua functions retrieve an additional first parameter compared to the C counterparts - a "state".
51 Most useful C functions and structures have lua FFI wrappers, sometimes with extra sugar.
52
53The modules follow the `Lua way <http://lua-users.org/wiki/ModuleDefinition>`_, where the module interface is returned in a named table.
54
55.. code-block:: lua
56
57 --- @module Count incoming queries
58 local counter = {}
59
60 function counter.init(module)
61 counter.total = 0
62 counter.last = 0
63 counter.failed = 0
64 end
65
66 function counter.deinit(module)
67 print('counted', counter.total, 'queries')
68 end
69
70 -- @function Run the q/s counter with given interval.
71 function counter.config(conf)
72 -- We can use the scripting facilities here
73 if counter.ev then event.cancel(counter.ev)
74 event.recurrent(conf.interval, function ()
75 print(counter.total - counter.last, 'q/s')
76 counter.last = counter.total
77 end)
78 end
79
80 return counter
81
82.. vv Hmm, we do not use these coroutine returns anywhere, so it's unclear whether they still work OK. Splitting work over time is now typically done via the ``event`` timers.
83
84.. The API functions may return an integer value just like in other languages, but they may also return a coroutine that will be continued asynchronously. A good use case for this approach is is a deferred initialization, e.g. loading a chunks of data or waiting for I/O.
85
86.. .. code-block:: lua
87
88 function counter.init(module)
89 counter.total = 0
90 counter.last = 0
91 counter.failed = 0
92 return coroutine.create(function ()
93 for line in io.lines('/etc/hosts') do
94 load(module, line)
95 coroutine.yield()
96 end
97 end)
98 end
99
100The created module can be then loaded just like any other module, except it isn't very useful since it
101doesn't provide any layer to capture events. The Lua module can however provide a processing layer, just
102:ref:`like its C counterpart <lib-layers>`.
103
104.. code-block:: lua
105
106 -- Notice it isn't a function, but a table of functions
107 counter.layer = {
108 begin = function (state, data)
109 counter.total = counter.total + 1
110 return state
111 end,
112 finish = function (state, req, answer)
113 if state == kres.FAIL then
114 counter.failed = counter.failed + 1
115 end
116 return state
117 end
118 }
119
120There is currently an additional "feature" in comparison to C layer functions:
121some functions do not get called at all if ``state == kres.FAIL``;
122see docs for details: :c:type:`kr_layer_api`.
123
124Since the modules are like any other Lua modules, you can interact with them through the CLI and and any interface.
125
126.. tip:: Module discovery: ``kres_modules.`` is prepended to the module name and lua search path is used on that.
127
128
129Writing a module in C
130=====================
131
132As almost all the functions are optional, the minimal module looks like this:
133
134.. code-block:: c
135
136 #include "lib/module.h"
137 /* Convenience macro to declare module ABI. */
138 KR_MODULE_EXPORT(mymodule)
139
140.. TODO it's probably not a good idea to start C module tutorial by pthread_create()
141
142Let's define an observer thread for the module as well. It's going to be stub for the sake of brevity,
143but you can for example create a condition, and notify the thread from query processing by declaring
144module layer (see the :ref:`Writing layers <lib-layers>`).
145
146.. code-block:: c
147
148 static void* observe(void *arg)
149 {
150 /* ... do some observing ... */
151 }
152
153 int mymodule_init(struct kr_module *module)
154 {
155 /* Create a thread and start it in the background. */
156 pthread_t thr_id;
157 int ret = pthread_create(&thr_id, NULL, &observe, NULL);
158 if (ret != 0) {
159 return kr_error(errno);
160 }
161
162 /* Keep it in the thread */
163 module->data = thr_id;
164 return kr_ok();
165 }
166
167 int mymodule_deinit(struct kr_module *module)
168 {
169 /* ... signalize cancellation ... */
170 void *res = NULL;
171 pthread_t thr_id = (pthread_t) module->data;
172 int ret = pthread_join(thr_id, res);
173 if (ret != 0) {
174 return kr_error(errno);
175 }
176
177 return kr_ok();
178 }
179
180This example shows how a module can run in the background, this enables you to, for example, observe
181and publish data about query resolution.
182
183Configuring modules
184===================
185
186There is a callback ``X_config()`` that you can implement, see hints module.
187
188.. _mod-properties:
189
190Exposing C module properties
191============================
192
193A module can offer NULL-terminated list of *properties*, each property is essentially a callable with free-form JSON input/output.
194JSON was chosen as an interchangeable format that doesn't require any schema beforehand, so you can do two things - query the module properties
195from external applications or between modules (e.g. `statistics` module can query `cache` module for memory usage).
196JSON was chosen not because it's the most efficient protocol, but because it's easy to read and write and interface to outside world.
197
198.. note:: The ``void *env`` is a generic module interface. Since we're implementing daemon modules, the pointer can be cast to ``struct engine*``.
199 This is guaranteed by the implemented API version (see `Writing a module in C`_).
200
201Here's an example how a module can expose its property:
202
203.. code-block:: c
204
205 char* get_size(void *env, struct kr_module *m,
206 const char *args)
207 {
208 /* Get cache from engine. */
209 struct engine *engine = env;
210 struct kr_cache *cache = &engine->resolver.cache;
211 /* Read item count */
212 int count = (cache->api)->count(cache->db);
213 char *result = NULL;
214 asprintf(&result, "{ \"result\": %d }", count);
215
216 return result;
217 }
218
219 struct kr_prop *cache_props(void)
220 {
221 static struct kr_prop prop_list[] = {
222 /* Callback, Name, Description */
223 {&get_size, "get_size", "Return number of records."},
224 {NULL, NULL, NULL}
225 };
226 return prop_list;
227 }
228
229 KR_MODULE_EXPORT(cache)
230
231Once you load the module, you can call the module property from the interactive console.
232*Note:* the JSON output will be transparently converted to Lua tables.
233
234.. code-block:: bash
235
236 $ kresd
237 ...
238 [system] started in interactive mode, type 'help()'
239 > modules.load('cached')
240 > cached.get_size()
241 [size] => 53
242
243.. No idea what this talks about, but kept for now:
244.. *Note:* this relies on function pointers, so the same ``static inline`` trick as for the ``Layer()`` is required for C.
245
246Special properties
247------------------
248
249If the module declares properties ``get`` or ``set``, they can be used in the Lua interpreter as
250regular tables.
251
252