1.\" $FreeBSD$ 2.\" 3.Dd October 23, 2016 4.Dt LIBRSS 3 5.Os 6.Sh NAME 7.Nm librss 8.Nd Provide Receive-side scaling awareness to userland applications 9.Sh LIBRARY 10.Lb librss 11.Sh SYNOPSIS 12.In sys/param.h 13.In sys/cpuset.h 14.In librss.h 15.Ft struct rss_config * 16.Fn rss_config_get "void" 17.Ft void 18.Fn rss_config_free "struct rss_config *cfg" 19.Ft int 20.Fn rss_config_get_bucket_count "struct rss_config *cfg" 21.Ft int 22.Fn rss_get_bucket_cpuset "struct rss_config *rc" "rss_bucket_type_t btype" "int bucket" "cpuset_t *cs" 23.Ft int 24.Fn rss_set_bucket_rebalance_cb "rss_bucket_rebalance_cb_t *cb" "void *cbdata" 25.Ft int 26.Fn rss_sock_set_recvrss "int fd" "int af" "int val" 27.Sh DESCRIPTION 28The 29.Nm 30library and the functions it provides are used for both fetching 31the system RSS configuration and interacting with RSS aware 32sockets. 33.Pp 34Applications will typically call 35.Fn rss_config_get 36to fetch the current RSS configuration from the system and perform 37initial setup. 38This typically involves spawning worker threads, one per RSS bucket, 39and optionally binding them to the per-bucket CPU set. 40.Pp 41The 42.Vt rss_config 43struct is defined as: 44.Bd -literal 45struct rss_config { 46 int rss_ncpus; 47 int rss_nbuckets; 48 int rss_basecpu; 49 int *rss_bucket_map; 50}; 51.Ed 52.Pp 53Applications will typically use the 54.Fn rss_config_get_bucket_count 55function to fetch the number of RSS buckets, create one thread 56per RSS bucket for RSS aware work, then one RSS aware socket to receive 57UDP datagrams or TCP connections 58in each particular RSS bucket / thread. 59.Pp 60The 61.Fn rss_get_bucket_cpuset 62function sets the given cpuset up for the given 63RSS bucket and behaviour. 64Typically applications will wish to just query for 65.Vt RSS_BUCKET_TYPE_KERNEL_ALL 66unless they wish to potentially setup different 67worker threads for transmit and receive. 68.Pp 69The 70.Vt rss_bucket_type_t 71enum is defined as: 72.Bd -literal 73typedef enum { 74 RSS_BUCKET_TYPE_NONE = 0, 75 RSS_BUCKET_TYPE_KERNEL_ALL = 1, 76 RSS_BUCKET_TYPE_KERNEL_TX = 2, 77 RSS_BUCKET_TYPE_KERNEL_RX = 3, 78 RSS_BUCKET_TYPE_MAX = 3, 79} rss_bucket_type_t; 80.Ed 81.Pp 82The rebalance callback 83.Vt rss_bucket_rebalance_cb_t 84is defined as: 85.Bd -literal 86typedef void rss_bucket_rebalance_cb_t(void *arg); 87.Ed 88.Pp 89The 90.Fn rss_set_bucket_rebalance_cb 91function sets an optional callback that will be called if the kernel 92rebalances RSS buckets. 93This is intended as a future expansion to rebalance buckets rather than 94reprogram the RSS key, so typically the only work to be performed 95is to rebind worker threads to an updated cpuset. 96.Pp 97Once RSS setup is completed, 98.Fn rss_config_free 99is called to free the RSS configuration structure. 100.Pp 101If 102.Vt val 103is set to 1, the socket can be placed in an RSS bucket and will only accept 104datagrams (for UDP) or connections (for TCP) that are received for that 105RSS bucket. 106If set to 0, the socket is placed in the default PCB and will see 107datagrams/connections that are not initially consumed by a PCB aware 108socket. 109.Pp 110The 111.Fn rss_sock_set_recvrss 112function enables or disables receiving RSS related information 113as socket options in. 114.2 recvmsg 115calls. 116.Pp 117When enabled, UDP datagrams will have a message with the 118.Vt IP_RECVFLOWID 119option indicating the 32-bit receive flowid as a uint32_t, 120and the 121.Vt IP_RECVRSSBUCKETID 122option indicating the 32 bit RSS bucket id as a uint32_t. 123.Sh ERRORS 124The functions return either <0 or NULL as appropriate upon error. 125.Sh HISTORY 126The 127.Xr librss.3 128library first appeared in 129.Fx 11.0 . 130.Sh AUTHORS 131.An Adrian Chadd Aq Mt adrian@FreeBSD.org 132.Sh BUGS 133There is currently no kernel mechanism to rebalance the RSS bucket to CPU 134mapping, and so the callback mechanism is a no-op. 135