1Threads 2======= 3 4Wait a minute? Why are we on threads? Aren't event loops supposed to be **the 5way** to do *web-scale programming*? Well... no. Threads are still the medium in 6which processors do their jobs. Threads are therefore mighty useful sometimes, even 7though you might have to wade through various synchronization primitives. 8 9Threads are used internally to fake the asynchronous nature of all of the system 10calls. libuv also uses threads to allow you, the application, to perform a task 11asynchronously that is actually blocking, by spawning a thread and collecting 12the result when it is done. 13 14Today there are two predominant thread libraries: the Windows threads 15implementation and POSIX's :man:`pthreads(7)`. libuv's thread API is analogous to 16the pthreads API and often has similar semantics. 17 18A notable aspect of libuv's thread facilities is that it is a self contained 19section within libuv. Whereas other features intimately depend on the event 20loop and callback principles, threads are complete agnostic, they block as 21required, signal errors directly via return values, and, as shown in the 22:ref:`first example <thread-create-example>`, don't even require a running 23event loop. 24 25libuv's thread API is also very limited since the semantics and syntax of 26threads are different on all platforms, with different levels of completeness. 27 28This chapter makes the following assumption: **There is only one event loop, 29running in one thread (the main thread)**. No other thread interacts 30with the event loop (except using ``uv_async_send``). 31 32Core thread operations 33---------------------- 34 35There isn't much here, you just start a thread using ``uv_thread_create()`` and 36wait for it to close using ``uv_thread_join()``. 37 38.. _thread-create-example: 39 40.. rubric:: thread-create/main.c 41.. literalinclude:: ../../code/thread-create/main.c 42 :linenos: 43 :lines: 26-36 44 :emphasize-lines: 3-7 45 46.. tip:: 47 48 ``uv_thread_t`` is just an alias for ``pthread_t`` on Unix, but this is an 49 implementation detail, avoid depending on it to always be true. 50 51The second parameter is the function which will serve as the entry point for 52the thread, the last parameter is a ``void *`` argument which can be used to pass 53custom parameters to the thread. The function ``hare`` will now run in a separate 54thread, scheduled pre-emptively by the operating system: 55 56.. rubric:: thread-create/main.c 57.. literalinclude:: ../../code/thread-create/main.c 58 :linenos: 59 :lines: 6-14 60 :emphasize-lines: 2 61 62Unlike ``pthread_join()`` which allows the target thread to pass back a value to 63the calling thread using a second parameter, ``uv_thread_join()`` does not. To 64send values use :ref:`inter-thread-communication`. 65 66Synchronization Primitives 67-------------------------- 68 69This section is purposely spartan. This book is not about threads, so I only 70catalogue any surprises in the libuv APIs here. For the rest you can look at 71the :man:`pthreads(7)` man pages. 72 73Mutexes 74~~~~~~~ 75 76The mutex functions are a **direct** map to the pthread equivalents. 77 78.. rubric:: libuv mutex functions 79.. code-block:: c 80 81 int uv_mutex_init(uv_mutex_t* handle); 82 int uv_mutex_init_recursive(uv_mutex_t* handle); 83 void uv_mutex_destroy(uv_mutex_t* handle); 84 void uv_mutex_lock(uv_mutex_t* handle); 85 int uv_mutex_trylock(uv_mutex_t* handle); 86 void uv_mutex_unlock(uv_mutex_t* handle); 87 88The ``uv_mutex_init()``, ``uv_mutex_init_recursive()`` and ``uv_mutex_trylock()`` 89functions will return 0 on success, and an error code otherwise. 90 91If `libuv` has been compiled with debugging enabled, ``uv_mutex_destroy()``, 92``uv_mutex_lock()`` and ``uv_mutex_unlock()`` will ``abort()`` on error. 93Similarly ``uv_mutex_trylock()`` will abort if the error is anything *other 94than* ``EAGAIN`` or ``EBUSY``. 95 96Recursive mutexes are supported, but you should not rely on them. Also, they 97should not be used with ``uv_cond_t`` variables. 98 99The default BSD mutex implementation will raise an error if a thread which has 100locked a mutex attempts to lock it again. For example, a construct like:: 101 102 uv_mutex_init(a_mutex); 103 uv_mutex_lock(a_mutex); 104 uv_thread_create(thread_id, entry, (void *)a_mutex); 105 uv_mutex_lock(a_mutex); 106 // more things here 107 108can be used to wait until another thread initializes some stuff and then 109unlocks ``a_mutex`` but will lead to your program crashing if in debug mode, or 110return an error in the second call to ``uv_mutex_lock()``. 111 112.. note:: 113 114 Mutexes on Windows are always recursive. 115 116Locks 117~~~~~ 118 119Read-write locks are a more granular access mechanism. Two readers can access 120shared memory at the same time. A writer may not acquire the lock when it is 121held by a reader. A reader or writer may not acquire a lock when a writer is 122holding it. Read-write locks are frequently used in databases. Here is a toy 123example. 124 125.. rubric:: locks/main.c - simple rwlocks 126.. literalinclude:: ../../code/locks/main.c 127 :linenos: 128 :emphasize-lines: 13,16,27,31,42,55 129 130Run this and observe how the readers will sometimes overlap. In case of 131multiple writers, schedulers will usually give them higher priority, so if you 132add two writers, you'll see that both writers tend to finish first before the 133readers get a chance again. 134 135We also use barriers in the above example so that the main thread can wait for 136all readers and writers to indicate they have ended. 137 138Others 139~~~~~~ 140 141libuv also supports semaphores_, `condition variables`_ and barriers_ with APIs 142very similar to their pthread counterparts. 143 144.. _semaphores: https://en.wikipedia.org/wiki/Semaphore_(programming) 145.. _condition variables: https://en.wikipedia.org/wiki/Monitor_(synchronization)#Condition_variables_2 146.. _barriers: https://en.wikipedia.org/wiki/Barrier_(computer_science) 147 148In addition, libuv provides a convenience function ``uv_once()``. Multiple 149threads can attempt to call ``uv_once()`` with a given guard and a function 150pointer, **only the first one will win, the function will be called once and 151only once**:: 152 153 /* Initialize guard */ 154 static uv_once_t once_only = UV_ONCE_INIT; 155 156 int i = 0; 157 158 void increment() { 159 i++; 160 } 161 162 void thread1() { 163 /* ... work */ 164 uv_once(once_only, increment); 165 } 166 167 void thread2() { 168 /* ... work */ 169 uv_once(once_only, increment); 170 } 171 172 int main() { 173 /* ... spawn threads */ 174 } 175 176After all threads are done, ``i == 1``. 177 178.. _libuv-work-queue: 179 180libuv v0.11.11 onwards also added a ``uv_key_t`` struct and api_ for 181thread-local storage. 182 183.. _api: http://docs.libuv.org/en/v1.x/threading.html#thread-local-storage 184 185libuv work queue 186---------------- 187 188``uv_queue_work()`` is a convenience function that allows an application to run 189a task in a separate thread, and have a callback that is triggered when the 190task is done. A seemingly simple function, what makes ``uv_queue_work()`` 191tempting is that it allows potentially any third-party libraries to be used 192with the event-loop paradigm. When you use event loops, it is *imperative to 193make sure that no function which runs periodically in the loop thread blocks 194when performing I/O or is a serious CPU hog*, because this means that the loop 195slows down and events are not being handled at full capacity. 196 197However, a lot of existing code out there features blocking functions (for example 198a routine which performs I/O under the hood) to be used with threads if you 199want responsiveness (the classic 'one thread per client' server model), and 200getting them to play with an event loop library generally involves rolling your 201own system of running the task in a separate thread. libuv just provides 202a convenient abstraction for this. 203 204Here is a simple example inspired by `node.js is cancer`_. We are going to 205calculate fibonacci numbers, sleeping a bit along the way, but run it in 206a separate thread so that the blocking and CPU bound task does not prevent the 207event loop from performing other activities. 208 209.. rubric:: queue-work/main.c - lazy fibonacci 210.. literalinclude:: ../../code/queue-work/main.c 211 :linenos: 212 :lines: 17-29 213 214The actual task function is simple, nothing to show that it is going to be 215run in a separate thread. The ``uv_work_t`` structure is the clue. You can pass 216arbitrary data through it using the ``void* data`` field and use it to 217communicate to and from the thread. But be sure you are using proper locks if 218you are changing things while both threads may be running. 219 220The trigger is ``uv_queue_work``: 221 222.. rubric:: queue-work/main.c 223.. literalinclude:: ../../code/queue-work/main.c 224 :linenos: 225 :lines: 31-44 226 :emphasize-lines: 10 227 228The thread function will be launched in a separate thread, passed the 229``uv_work_t`` structure and once the function returns, the *after* function 230will be called on the thread the event loop is running in. It will be passed 231the same structure. 232 233For writing wrappers to blocking libraries, a common :ref:`pattern <baton>` 234is to use a baton to exchange data. 235 236Since libuv version `0.9.4` an additional function, ``uv_cancel()``, is 237available. This allows you to cancel tasks on the libuv work queue. Only tasks 238that *are yet to be started* can be cancelled. If a task has *already started 239executing, or it has finished executing*, ``uv_cancel()`` **will fail**. 240 241``uv_cancel()`` is useful to cleanup pending tasks if the user requests 242termination. For example, a music player may queue up multiple directories to 243be scanned for audio files. If the user terminates the program, it should quit 244quickly and not wait until all pending requests are run. 245 246Let's modify the fibonacci example to demonstrate ``uv_cancel()``. We first set 247up a signal handler for termination. 248 249.. rubric:: queue-cancel/main.c 250.. literalinclude:: ../../code/queue-cancel/main.c 251 :linenos: 252 :lines: 43- 253 254When the user triggers the signal by pressing ``Ctrl+C`` we send 255``uv_cancel()`` to all the workers. ``uv_cancel()`` will return ``0`` for those that are already executing or finished. 256 257.. rubric:: queue-cancel/main.c 258.. literalinclude:: ../../code/queue-cancel/main.c 259 :linenos: 260 :lines: 33-41 261 :emphasize-lines: 6 262 263For tasks that do get cancelled successfully, the *after* function is called 264with ``status`` set to ``UV_ECANCELED``. 265 266.. rubric:: queue-cancel/main.c 267.. literalinclude:: ../../code/queue-cancel/main.c 268 :linenos: 269 :lines: 28-31 270 :emphasize-lines: 2 271 272``uv_cancel()`` can also be used with ``uv_fs_t`` and ``uv_getaddrinfo_t`` 273requests. For the filesystem family of functions, ``uv_fs_t.errorno`` will be 274set to ``UV_ECANCELED``. 275 276.. TIP:: 277 278 A well designed program would have a way to terminate long running workers 279 that have already started executing. Such a worker could periodically check 280 for a variable that only the main process sets to signal termination. 281 282.. _inter-thread-communication: 283 284Inter-thread communication 285-------------------------- 286 287Sometimes you want various threads to actually send each other messages *while* 288they are running. For example you might be running some long duration task in 289a separate thread (perhaps using ``uv_queue_work``) but want to notify progress 290to the main thread. This is a simple example of having a download manager 291informing the user of the status of running downloads. 292 293.. rubric:: progress/main.c 294.. literalinclude:: ../../code/progress/main.c 295 :linenos: 296 :lines: 7-8,34- 297 :emphasize-lines: 2,11 298 299The async thread communication works *on loops* so although any thread can be 300the message sender, only threads with libuv loops can be receivers (or rather 301the loop is the receiver). libuv will invoke the callback (``print_progress``) 302with the async watcher whenever it receives a message. 303 304.. warning:: 305 306 It is important to realize that since the message send is *async*, the callback 307 may be invoked immediately after ``uv_async_send`` is called in another 308 thread, or it may be invoked after some time. libuv may also combine 309 multiple calls to ``uv_async_send`` and invoke your callback only once. The 310 only guarantee that libuv makes is -- The callback function is called *at 311 least once* after the call to ``uv_async_send``. If you have no pending 312 calls to ``uv_async_send``, the callback won't be called. If you make two 313 or more calls, and libuv hasn't had a chance to run the callback yet, it 314 *may* invoke your callback *only once* for the multiple invocations of 315 ``uv_async_send``. Your callback will never be called twice for just one 316 event. 317 318.. rubric:: progress/main.c 319.. literalinclude:: ../../code/progress/main.c 320 :linenos: 321 :lines: 10-23 322 :emphasize-lines: 7-8 323 324In the download function, we modify the progress indicator and queue the message 325for delivery with ``uv_async_send``. Remember: ``uv_async_send`` is also 326non-blocking and will return immediately. 327 328.. rubric:: progress/main.c 329.. literalinclude:: ../../code/progress/main.c 330 :linenos: 331 :lines: 30-33 332 333The callback is a standard libuv pattern, extracting the data from the watcher. 334 335Finally it is important to remember to clean up the watcher. 336 337.. rubric:: progress/main.c 338.. literalinclude:: ../../code/progress/main.c 339 :linenos: 340 :lines: 25-28 341 :emphasize-lines: 3 342 343After this example, which showed the abuse of the ``data`` field, bnoordhuis_ 344pointed out that using the ``data`` field is not thread safe, and 345``uv_async_send()`` is actually only meant to wake up the event loop. Use 346a mutex or rwlock to ensure accesses are performed in the right order. 347 348.. note:: 349 350 mutexes and rwlocks **DO NOT** work inside a signal handler, whereas 351 ``uv_async_send`` does. 352 353One use case where ``uv_async_send`` is required is when interoperating with 354libraries that require thread affinity for their functionality. For example in 355node.js, a v8 engine instance, contexts and its objects are bound to the thread 356that the v8 instance was started in. Interacting with v8 data structures from 357another thread can lead to undefined results. Now consider some node.js module 358which binds a third party library. It may go something like this: 359 3601. In node, the third party library is set up with a JavaScript callback to be 361 invoked for more information:: 362 363 var lib = require('lib'); 364 lib.on_progress(function() { 365 console.log("Progress"); 366 }); 367 368 lib.do(); 369 370 // do other stuff 371 3722. ``lib.do`` is supposed to be non-blocking but the third party lib is 373 blocking, so the binding uses ``uv_queue_work``. 374 3753. The actual work being done in a separate thread wants to invoke the progress 376 callback, but cannot directly call into v8 to interact with JavaScript. So 377 it uses ``uv_async_send``. 378 3794. The async callback, invoked in the main loop thread, which is the v8 thread, 380 then interacts with v8 to invoke the JavaScript callback. 381 382---- 383 384.. _node.js is cancer: http://widgetsandshit.com/teddziuba/2011/10/node-js-is-cancer.html 385.. _bnoordhuis: https://github.com/bnoordhuis 386