1.\" 2.\" Copyright (c) 2012 The DragonFly Project. All rights reserved. 3.\" 4.\" This code is derived from software contributed to The DragonFly Project 5.\" by Nuno Antunes <nuno.antunes@gmail.com>. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in 15.\" the documentation and/or other materials provided with the 16.\" distribution. 17.\" 3. Neither the name of The DragonFly Project nor the names of its 18.\" contributors may be used to endorse or promote products derived 19.\" from this software without specific, prior written permission. 20.\" 21.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 22.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 23.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 24.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 25.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 26.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 27.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 28.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 29.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 30.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 31.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 32.\" SUCH DAMAGE. 33.\" 34.Dd September 17, 2012 35.Dt MSGPORT 9 36.Os 37.Sh NAME 38.Nm lwkt_initport_thread , 39.Nm lwkt_initport_spin , 40.Nm lwkt_initport_serialize , 41.Nm lwkt_initport_panic , 42.Nm lwkt_initport_replyonly_null , 43.Nm lwkt_initport_replyonly , 44.Nm lwkt_initport_putonly , 45.Nm lwkt_sendmsg , 46.Nm lwkt_domsg , 47.Nm lwkt_forwardmsg , 48.Nm lwkt_abortmsg , 49.Nm lwkt_initmsg , 50.Nm lwkt_initmsg_abortable , 51.Nm lwkt_beginmsg , 52.Nm lwkt_replymsg , 53.Nm lwkt_getport , 54.Nm lwkt_waitport , 55.Nm lwkt_waitmsg , 56.Nm lwkt_checkmsg , 57.Nm lwkt_dropmsg 58.Nd LWKT message passing interface 59.Sh SYNOPSIS 60.In sys/msgport.h 61.Ft void 62.Fn lwkt_initport_thread "lwkt_port_t port" "struct thread *td" 63.Ft void 64.Fn lwkt_initport_spin "lwkt_port_t port" "struct thread *td" 65.Ft void 66.Fn lwkt_initport_serialize "lwkt_port_t port" "struct lwkt_serialize *slz" 67.Ft void 68.Fn lwkt_initport_panic "lwkt_port_t port" 69.Ft void 70.Fn lwkt_initport_replyonly_null "lwkt_port_t port" 71.Ft void 72.Fn lwkt_initport_replyonly "lwkt_port_t port" "void (*rportfn)(lwkt_port_t, lwkt_msg_t)" 73.Ft void 74.Fn lwkt_initport_putonly "lwkt_port_t port" "int (*pportfn)(lwkt_port_t, lwkt_msg_t)" 75.Ft void 76.Fn lwkt_sendmsg "lwkt_port_t port" "lwkt_msg_t msg" 77.Ft int 78.Fn lwkt_domsg "lwkt_port_t port" "lwkt_msg_t msg" "int flags" 79.Ft int 80.Fn lwkt_forwardmsg "lwkt_port_t port" "lwkt_msg_t msg" 81.Ft void 82.Fn lwkt_abortmsg "lwkt_msg_t msg" 83.In sys/msgport2.h 84.Ft void 85.Fn lwkt_initmsg "lwkt_msg_t msg" "lwkt_port_t rport" "int flags" 86.Ft void 87.Fn lwkt_initmsg_abortable "lwkt_msg_t msg" "lwkt_port_t rport" "int flags" "void (*abortfn)(lwkt_msg_t)" 88.Ft int 89.Fn lwkt_beginmsg "lwkt_port_t port" "lwkt_msg_t msg" 90.Ft void 91.Fn lwkt_replymsg "lwkt_msg_t msg" "int error" 92.Ft void * 93.Fn lwkt_getport "lwkt_port_t port" 94.Ft void * 95.Fn lwkt_waitport "lwkt_port_t port" "int flags" 96.Ft int 97.Fn lwkt_waitmsg "lwkt_msg_t msg" "int flags" 98.Ft int 99.Fn lwkt_checkmsg "lwkt_msg_t msg" 100.Ft void 101.Fn lwkt_dropmsg "lwkt_msg_t msg" 102.Sh DESCRIPTION 103Light weight kernel threads in 104.Dx 105may use a message passing interface to communicate with each other. 106Messages are sent to message ports. 107All light weight kernel threads have a built-in message port, but you may create 108additional ports if necessary. 109The following types of message ports are available: 110.Bl -bullet 111.It 112thread ports 113.It 114spin ports 115.It 116serializer ports 117.El 118.Pp 119Ports of type 120.Sq thread 121are owned by a single light weight kernel thread. 122When a message is sent to a port of type 123.Sq thread , 124only the owner of that port is allowed to retrieve the message from it. 125When a message is sent to a port of type 126.Sq spin 127or to a port of type 128.Sq serializer , 129multiple threads are allowed to check that port for new 130messages and compete to retrieve them. 131You define the port type when you initialize the port. 132By default, the built-in port of every light weight kernel thread is 133automatically initialized to type 134.Sq thread . 135.Pp 136When a message is sent, the receiver should normally send back a reply. 137The reply is sent to the reply port that is registered on the original message. 138Messages can be replied to synchronously or asynchronously. 139The sender may request a synchronous or asynchronous reply to the message, 140however the target port will ultimately decide how the message will be treated. 141.Sh MESSAGE FUNCTIONS 142Messages must be initialized before being used. 143The 144.Fn lwkt_initmsg 145function initializes a message. 146The 147.Fa rport 148argument identifies the reply port which will be used for asynchronous replies. 149The 150.Fa flags 151argument sets any required flags for this message. 152Flags passed this way will simply be or'ed to any already existing flags on the 153message. 154.Pp 155The 156.Fn lwkt_initmsg_abortable 157function is similar to 158.Fn lwkt_initmsg 159but it takes an additional parameter 160.Fa abortfn 161which defines the abort function for this message. 162.Pp 163The 164.Fn lwkt_sendmsg 165function requests an asynchronous reply, sends the message and returns 166immediately. 167Under normal circumstances, users of this function may always expect the reply 168to be queued to the reply port registered on the message. 169The 170.Fa port 171argument defines the target port to which the 172.Fa msg 173message will be sent. 174.Pp 175The 176.Fn lwkt_domsg 177function requests a synchronous reply, sends the message and does not return 178until the message has been replied to. 179If the target port supports synchronous reply, this function will return that 180reply immediately. 181If not, and this is the most common case, this function will block and wait for the 182reply to arrive and then return it. 183The 184.Fa port 185argument defines the target port to which the 186.Fa msg 187message will be sent. 188.Pp 189The 190.Fn lwkt_replymsg 191function replies to a message that was processed asynchronously by the target 192port. 193This function is used by the thread on the receiving side. 194The 195.Fa msg 196argument is the message being replied to and the 197.Fa error 198argument is the actual response to send back. 199.Pp 200The 201.Fn lwkt_forwardmsg 202simply forwards a message to another port. 203The 204.Fa port 205argument defines the target port to which the 206.Fa msg 207message will be sent. 208.Pp 209If a message has been initialized as abortable, you can use the 210.Fn lwkt_abortmsg 211function to try to abort it. 212The 213.Fa abortfn 214passed upon the initialisation with 215.Fn lwkt_initmsg_abortable 216will be called by this function. 217.Pp 218The 219.Fn lwkt_dropmsg 220will dequeue the specified message from the target port it was sent to and makes 221it look like it was never sent. 222This function can only be used by the thread that owns the target port. 223.Sh PORT FUNCTIONS 224The 225.Fn lwkt_initport_thread 226initializes the specified 227.Fa port 228with the default 229.Sq thread 230port type handlers. 231The 232.Fa td 233argument defines the owner thread of the port and only that thread is allowed to 234receive messages on it. 235.Pp 236The 237.Fn lwkt_initport_spin 238initializes the specified 239.Fa port 240with the default 241.Sq spin 242port type handlers. 243The 244.Fa td 245argument defines the owner thread of the port, for cases where thread built-in 246ports are initialized as 247.Sq spin 248ports. 249If 250.Dv NULL 251is passed, then the port will not have a defined owner, so functions like 252.Fn lwkt_dropmsg 253will not be available for this port. 254This function will also initialize the embedded spinlock within the 255.Vt lwkt_port 256structure which will protect subsequent port access. 257.Pp 258The 259.Fn lwkt_initport_serialize 260function initializes the specified 261.Fa port 262with the default 263.Sq serializer 264port type handlers. 265The subsequent port access will be protected by the passed 266.Fa slz 267serializer lock. 268.Pp 269The 270.Fn lwkt_getport 271function checks the specified 272.Fa port 273for available messages, dequeues the first one and returns it. 274If no messages are available then 275.Dv NULL 276is returned instead. 277This function is used by threads on the receiving side. 278.Pp 279The 280.Fn lwkt_waitport 281function checks the specified 282.Fa port 283for available messages, dequeues the first one and returns it. 284If no messages are available then the caller thread will sleep until a message 285arrives on the specified port. 286The 287.Fa flags 288argument defines the flags used for the sleep. 289This function is used by threads on the receiving side. 290.Sh SPECIAL PORT INITIALIZERS 291The 292.Fn lwkt_initport_replyonly 293function initializes a 294.Fa port 295which is used only as reply port and may have a custom reply port handler. 296The reply port handler is specified with the 297.Fa rportfn 298argument. 299All the other handlers will panic the system if they are called. 300This initializer is normally used on ports for freeing resources after the 301messages have fulfilled their purpose. 302.Pp 303The 304.Fn lwkt_initport_replyonly_null 305function initializes a 306.Fa port 307which is used only as reply port. 308The reply port handler will simply mark the message as being done and will not 309attempt to queue it. 310All the other handlers will panic the system if they are called. 311.Pp 312The 313.Fn lwkt_initport_putonly 314function initializes a 315.Fa port 316which is used only as target port. 317The putport handler is specified with the 318.Fa pportfn 319argument. 320All the other handlers will panic the system if they are called. 321.Pp 322The 323.Fn lwkt_initport_panic 324function initializes a 325.Fa port 326which will panic the system if any of its handlers are called. 327This function is sometimes used to initialize a reply-only port which does not 328expect the messages to be replied to, e.g.\& when the messages should be 329consumed by the receiving thread and never replied back. 330.Sh INTERNAL MESSAGE FUNCTIONS 331The following functions are used only by the infrastructure, you should not 332need to use them directly unless in very rare cases. 333.Pp 334The 335.Fn lwkt_beginmsg 336function simply calls the target port's putport handler. 337This function is only called by the 338.Fn lwkt_sendmsg 339and 340.Fn lwkt_replymsg 341functions. 342The putport handler returns 343.Er EASYNC 344for messages processed asynchronously or any other value for messages processed 345synchronously. 346That return value of the putport handler is propagated by this function. 347The 348.Fa port 349argument defines the target port to which the 350.Fa msg 351message will be sent. 352.Pp 353The 354.Fn lwkt_waitmsg 355function puts the caller to sleep until the specified 356.Fa msg 357message has been replied to. 358The 359.Fa flags 360argument defines the flags used for the sleep. 361.Sh FILES 362The LWKT msgport implementation resides in 363.Pa sys/kern/lwkt_msgport.c . 364.Sh EXAMPLES 365.Bd -literal 366/* 367 * Example 1: per CPU threads. 368 * 369 */ 370 371#include <sys/thread.h> 372#include <sys/msgport.h> 373#include <sys/msgport2.h> 374 375static void my_service_loop(void *dummy); 376lwkt_port_t my_service_portfn(int cpu); 377void my_service_sendmsg(lwkt_msg_t lmsg, int cpu); 378int my_service_domsg(lwkt_msg_t lmsg, int cpu); 379 380/* Array of per-CPU target ports */ 381struct lwkt_port *my_service_ports[MAXCPU]; 382 383/* 384 * Create per-cpu threads for handling msg processing. Remember that built-in 385 * lwkt ports are automatically initialized to type 'thread' so we don't need 386 * to initialize them explicitly. 387 */ 388static void 389my_per_cpu_service_init(void) 390{ 391 int i; 392 thread_t td; 393 394 for (i = 0; i < ncpus; ++i) { 395 lwkt_create(my_service_loop, NULL, &td, 396 NULL, 0, i, "myservice_cpu %d", i); 397 my_service_ports[i] = &td->td_msgport; 398 } 399} 400 401/* 402 * This is the routine executed by the service threads on each CPU. 403 */ 404static void 405my_service_loop(void *dummy __unused) 406{ 407 lwkt_msg_t msg; 408 thread_t td = curthread; 409 int cpu = curthread->td_gd->gd_cpuid; 410 411 while ((msg = lwkt_waitport(&td->td_msgport, 0)) != NULL) { 412 /* Do some work in the receiver thread context. */ 413 kprintf("Received message on CPU %d.\en", cpu); 414 415 /* And finally reply to the message. */ 416 lwkt_replymsg(msg, 0); 417 } 418} 419 420/* 421 * Given a CPU id, return our respective service port. 422 */ 423__inline lwkt_port_t 424my_service_portfn(int cpu) 425{ 426 return my_service_ports[cpu]; 427} 428 429/* 430 * Send an asynchronous message to the service thread on a specific CPU. 431 */ 432void 433my_service_sendmsg(lwkt_msg_t lmsg, int cpu) 434{ 435 KKASSERT(cpu < ncpus); 436 lwkt_sendmsg(my_service_portfn(cpu), lmsg); 437} 438 439/* 440 * Send a synchronous message to the service thread on a specific CPU. 441 */ 442int 443my_service_domsg(lwkt_msg_t lmsg, int cpu) 444{ 445 KKASSERT(cpu < ncpus); 446 return lwkt_domsg(my_service_portfn(cpu), lmsg, 0); 447} 448 449/* 450 * Example use case. Initialize the service threads and send each one a 451 * message. 452 */ 453static void 454mod_load(void) 455{ 456 lwkt_msg lmsg; 457 lwkt_port_t builtin_port = &curthread->td_msgport; 458 int i; 459 460 my_per_cpu_service_init(); 461 for (i=0; i<ncpus; ++i) { 462 kprintf("Sending msg to CPU %d.\en", i); 463 lwkt_initmsg(&lmsg, builtin_port, 0); 464 my_service_domsg(&lmsg, i); 465 } 466} 467 468/* 469 * Example 2: Dynamic allocated message passing with automatic free. 470 * 471 * This scenario is used when resources need to be freed after the message 472 * has been replied to. Features: 473 * - An argument is passed within the message. 474 * - Messages are allocated with kmalloc(). Replying to the msg, kfree()s it. 475 */ 476 477#include <sys/thread.h> 478#include <sys/msgport.h> 479#include <sys/msgport2.h> 480 481void my_service_queue(void *arg); 482 483lwkt_port my_autofree_rport; 484lwkt_port_t my_service_port; 485 486/* 487 * Use this function to send messages with a void * argument to our 488 * service thread. 489 */ 490void 491my_service_queue(void *arg) 492{ 493 lwkt_msg_t msg; 494 495 msg = kmalloc(sizeof(*msg), M_TEMP, M_WAITOK); 496 497 /* Set reply port to autofree. */ 498 lwkt_initmsg(msg, &my_autofree_rport, 0); 499 500 /* Attach the argument to the message. */ 501 msg->u.ms_resultp = arg; 502 503 /* Send it. */ 504 lwkt_sendmsg(my_service_port, msg); 505} 506 507/* 508 * This is the routine executed by our service thread. 509 */ 510static void 511my_service_loop(void *dummy __unused) 512{ 513 lwkt_msg_t msg; 514 thread_t td = curthread; 515 516 while ((msg = lwkt_waitport(&td->td_msgport, 0)) != NULL) { 517 /* 518 * Do some work in the receiver thread context. In this 519 * example, the sender wrote his name in the argument he 520 * sent us. We print it here. 521 */ 522 char *arg = msg->u.ms_resultp; 523 kprintf("%s: Hi %s! Got your msg.\en", curthread->td_comm, arg); 524 525 /* And finally reply to the message. */ 526 lwkt_replymsg(msg, 0); 527 } 528} 529 530static void 531my_autofree_reply(lwkt_port_t port, lwkt_msg_t msg) 532{ 533 kfree(msg->u.ms_resultp, M_TEMP); 534 kfree(msg, M_TEMP); 535} 536 537static void 538my_service_init(void) 539{ 540 thread_t tdp; 541 542 /* Initialize our auto free reply port. */ 543 lwkt_initport_replyonly(&my_autofree_rport, my_autofree_reply); 544 545 /* Create our service thread on CPU 0. */ 546 lwkt_create(my_service_loop, NULL, &tdp, NULL, 0, 0, "myservice"); 547 my_service_port = &tdp->td_msgport; 548} 549 550/* 551 * Example use case. Initialize the service and send the current thread name 552 * to the service thread. 553 */ 554static void 555mod_load(void) 556{ 557 void *arg; 558 int len; 559 560 my_service_init(); 561 len = strlen(curthread->td_comm); 562 arg = kmalloc(len + 1, M_TEMP, M_WAITOK); 563 bcopy(curthread->td_comm, arg, len + 1); 564 kprintf("%s: Sending message.\en", curthread->td_comm); 565 my_service_queue(arg); 566} 567.Ed 568.Sh NOTES 569All the default putport handlers (used when a message is sent) currently 570implement asynchronous putports only, i.e.\& all 571.Fn *_putport 572handlers return 573.Er EASYNC . 574You can still have synchronous putport handlers (which are run in the sender's 575context) but you have to implement the function yourself and then override the 576default. 577.Pp 578Port handler functions can be overridden with custom functions if required. 579You can override the default putport handler by either using the 580.Fn lwkt_initport_putonly 581initializer, or by manipulating the mp_putport handler pointer directly on the 582.Vt lwkt_port 583structure. 584.Pp 585There is one such case where the putport handler is overridden in 586.Pa sys/net/netisr.c . 587In that case, the putport handler is overridden to detect a loopback message 588(when the target port belongs to the sending thread). 589This special putport handler turns the sent message into a direct function call 590instead of queueing it to the port. 591.Pp 592The 593.Fn lwkt_replymsg 594function works differently depending on the original message request. 595If the 596message was originally an asynchronous request, the reply will be queued to the 597sender's reply port. 598If the message was originally a synchronous request, then 599this function will just write the error response on the message and wake up the 600waiter without queueing the message to the reply port. 601There is no need to queue in the synchronous request case because the original 602sender had blocked waiting on this specific message with 603.Fn lwkt_domsg . 604.Pp 605As is the case with putport handler, the replyport handler can also be 606overridden. 607You override the default replyport handler by using the 608.Fn lwkt_initport_replyonly 609or the 610.Fn lwkt_initport_replyonly_null 611port initializers, or by manipulating the mp_replyport handler pointer directly 612on the 613.Vt lwkt_port 614structure. 615.Pp 616The sent message structure is reused for replies. 617When a message is replied to, the error response is written on the message 618which is subsequently sent to the reply port. 619.Sh SEE ALSO 620.Xr serializer 9 , 621.Xr sleep 9 , 622.Xr spinlock 9 623.Sh HISTORY 624The LWKT msgport interface first appeared in 625.Dx 1.0 . 626.Sh AUTHORS 627.An -nosplit 628The 629.Nm msgport 630message passing interface implementation was written by 631.An Matthew Dillon . 632This manual page was written by 633.An Nuno Antunes . 634