1.\" Copyright (c) 2001-2003 International Computer Science Institute 2.\" 3.\" Permission is hereby granted, free of charge, to any person obtaining a 4.\" copy of this software and associated documentation files (the "Software"), 5.\" to deal in the Software without restriction, including without limitation 6.\" the rights to use, copy, modify, merge, publish, distribute, sublicense, 7.\" and/or sell copies of the Software, and to permit persons to whom the 8.\" Software is furnished to do so, subject to the following conditions: 9.\" 10.\" The above copyright notice and this permission notice shall be included in 11.\" all copies or substantial portions of the Software. 12.\" 13.\" The names and trademarks of copyright holders may not be used in 14.\" advertising or publicity pertaining to the software without specific 15.\" prior permission. Title to copyright in this software and any associated 16.\" documentation will at all times remain with the copyright holders. 17.\" 18.\" THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19.\" IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20.\" FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21.\" AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22.\" LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 23.\" FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 24.\" DEALINGS IN THE SOFTWARE. 25.\" 26.\" $FreeBSD: src/share/man/man4/multicast.4,v 1.4 2004/07/09 09:22:36 ru Exp $ 27.\" $NetBSD: multicast.4,v 1.5 2010/12/23 19:15:55 njoly Exp $ 28.\" 29.Dd September 4, 2003 30.Dt MULTICAST 4 31.Os 32.\" 33.Sh NAME 34.Nm multicast 35.Nd Multicast Routing 36.\" 37.Sh SYNOPSIS 38.Cd "options MROUTING" 39.Pp 40.In sys/types.h 41.In sys/socket.h 42.In netinet/in.h 43.In netinet/ip_mroute.h 44.In netinet6/ip6_mroute.h 45.Ft int 46.Fn getsockopt "int s" IPPROTO_IP MRT_INIT "void *optval" "socklen_t *optlen" 47.Ft int 48.Fn setsockopt "int s" IPPROTO_IP MRT_INIT "const void *optval" "socklen_t optlen" 49.Ft int 50.Fn getsockopt "int s" IPPROTO_IPV6 MRT6_INIT "void *optval" "socklen_t *optlen" 51.Ft int 52.Fn setsockopt "int s" IPPROTO_IPV6 MRT6_INIT "const void *optval" "socklen_t optlen" 53.Sh DESCRIPTION 54.Tn "Multicast routing" 55is used to efficiently propagate data 56packets to a set of multicast listeners in multipoint networks. 57If unicast is used to replicate the data to all listeners, 58then some of the network links may carry multiple copies of the same 59data packets. 60With multicast routing, the overhead is reduced to one copy 61(at most) per network link. 62.Pp 63All multicast-capable routers must run a common multicast routing 64protocol. 65The Distance Vector Multicast Routing Protocol (DVMRP) 66was the first developed multicast routing protocol. 67Later, other protocols such as Multicast Extensions to OSPF (MOSPF), 68Core Based Trees (CBT), 69Protocol Independent Multicast - Sparse Mode (PIM-SM), 70and Protocol Independent Multicast - Dense Mode (PIM-DM) 71were developed as well. 72.Pp 73To start multicast routing, 74the user must enable multicast forwarding in the kernel 75(see 76.Sx SYNOPSIS 77about the kernel configuration options), 78and must run a multicast routing capable user-level process. 79From developer's point of view, 80the programming guide described in the 81.Sx Programming Guide 82section should be used to control the multicast forwarding in the kernel. 83.\" 84.Ss Programming Guide 85This section provides information about the basic multicast routing API. 86The so-called 87.Dq advanced multicast API 88is described in the 89.Sx Advanced Multicast API Programming Guide 90section. 91.Pp 92First, a multicast routing socket must be open. 93That socket would be used 94to control the multicast forwarding in the kernel. 95Note that most operations below require certain privilege 96(i.e., root privilege): 97.Bd -literal 98/* IPv4 */ 99int mrouter_s4; 100mrouter_s4 = socket(AF_INET, SOCK_RAW, IPPROTO_IGMP); 101.Ed 102.Bd -literal 103int mrouter_s6; 104mrouter_s6 = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); 105.Ed 106.Pp 107Note that if the router needs to open an IGMP or ICMPv6 socket 108(in case of IPv4 and IPv6 respectively) 109for sending or receiving of IGMP or MLD multicast group membership messages, 110then the same 111.Va mrouter_s4 112or 113.Va mrouter_s6 114sockets should be used 115for sending and receiving respectively IGMP or MLD messages. 116In case of 117.Bx Ns 118-derived kernel, it may be possible to open separate sockets 119for IGMP or MLD messages only. 120However, some other kernels (e.g., 121.Tn Linux ) 122require that the multicast 123routing socket must be used for sending and receiving of IGMP or MLD 124messages. 125Therefore, for portability reason the multicast 126routing socket should be reused for IGMP and MLD messages as well. 127.Pp 128After the multicast routing socket is open, it can be used to enable 129or disable multicast forwarding in the kernel: 130.Bd -literal 131/* IPv4 */ 132int v = 1; /* 1 to enable, or 0 to disable */ 133setsockopt(mrouter_s4, IPPROTO_IP, MRT_INIT, (void *)\*[Am]v, sizeof(v)); 134.Ed 135.Bd -literal 136/* IPv6 */ 137int v = 1; /* 1 to enable, or 0 to disable */ 138setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_INIT, (void *)\*[Am]v, sizeof(v)); 139\&... 140/* If necessary, filter all ICMPv6 messages */ 141struct icmp6_filter filter; 142ICMP6_FILTER_SETBLOCKALL(\*[Am]filter); 143setsockopt(mrouter_s6, IPPROTO_ICMPV6, ICMP6_FILTER, (void *)\*[Am]filter, 144 sizeof(filter)); 145.Ed 146.Pp 147After multicast forwarding is enabled, the multicast routing socket 148can be used to enable PIM processing in the kernel if we are running PIM-SM or 149PIM-DM 150(see 151.Xr pim 4 ) . 152.Pp 153For each network interface (e.g., physical or a virtual tunnel) 154that would be used for multicast forwarding, a corresponding 155multicast interface must be added to the kernel: 156.Bd -literal 157/* IPv4 */ 158struct vifctl vc; 159memset(\*[Am]vc, 0, sizeof(vc)); 160/* Assign all vifctl fields as appropriate */ 161vc.vifc_vifi = vif_index; 162vc.vifc_flags = vif_flags; 163vc.vifc_threshold = min_ttl_threshold; 164vc.vifc_rate_limit = max_rate_limit; 165memcpy(\*[Am]vc.vifc_lcl_addr, \*[Am]vif_local_address, sizeof(vc.vifc_lcl_addr)); 166if (vc.vifc_flags \*[Am] VIFF_TUNNEL) 167 memcpy(\*[Am]vc.vifc_rmt_addr, \*[Am]vif_remote_address, 168 sizeof(vc.vifc_rmt_addr)); 169setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_VIF, (void *)\*[Am]vc, 170 sizeof(vc)); 171.Ed 172.Pp 173The 174.Va vif_index 175must be unique per vif. 176The 177.Va vif_flags 178contains the 179.Dv VIFF_* 180flags as defined in 181.In netinet/ip_mroute.h . 182The 183.Va min_ttl_threshold 184contains the minimum TTL a multicast data packet must have to be 185forwarded on that vif. 186Typically, it would have value of 1. 187The 188.Va max_rate_limit 189contains the maximum rate (in bits/s) of the multicast data packets forwarded 190on that vif. 191Value of 0 means no limit. 192The 193.Va vif_local_address 194contains the local IP address of the corresponding local interface. 195The 196.Va vif_remote_address 197contains the remote IP address in case of DVMRP multicast tunnels. 198.Bd -literal 199/* IPv6 */ 200struct mif6ctl mc; 201memset(\*[Am]mc, 0, sizeof(mc)); 202/* Assign all mif6ctl fields as appropriate */ 203mc.mif6c_mifi = mif_index; 204mc.mif6c_flags = mif_flags; 205mc.mif6c_pifi = pif_index; 206setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_ADD_MIF, (void *)\*[Am]mc, 207 sizeof(mc)); 208.Ed 209.Pp 210The 211.Va mif_index 212must be unique per vif. 213The 214.Va mif_flags 215contains the 216.Dv MIFF_* 217flags as defined in 218.In netinet6/ip6_mroute.h . 219The 220.Va pif_index 221is the physical interface index of the corresponding local interface. 222.Pp 223A multicast interface is deleted by: 224.Bd -literal 225/* IPv4 */ 226vifi_t vifi = vif_index; 227setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_VIF, (void *)\*[Am]vifi, 228 sizeof(vifi)); 229.Ed 230.Bd -literal 231/* IPv6 */ 232mifi_t mifi = mif_index; 233setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_DEL_MIF, (void *)\*[Am]mifi, 234 sizeof(mifi)); 235.Ed 236.Pp 237After the multicast forwarding is enabled, and the multicast virtual 238interfaces are 239added, the kernel may deliver upcall messages (also called signals 240later in this text) on the multicast routing socket that was open 241earlier with 242.Dv MRT_INIT 243or 244.Dv MRT6_INIT . 245The IPv4 upcalls have 246.Vt "struct igmpmsg" 247header (see 248.In netinet/ip_mroute.h ) 249with field 250.Va im_mbz 251set to zero. 252Note that this header follows the structure of 253.Vt "struct ip" 254with the protocol field 255.Va ip_p 256set to zero. 257The IPv6 upcalls have 258.Vt "struct mrt6msg" 259header (see 260.In netinet6/ip6_mroute.h ) 261with field 262.Va im6_mbz 263set to zero. 264Note that this header follows the structure of 265.Vt "struct ip6_hdr" 266with the next header field 267.Va ip6_nxt 268set to zero. 269.Pp 270The upcall header contains field 271.Va im_msgtype 272and 273.Va im6_msgtype 274with the type of the upcall 275.Dv IGMPMSG_* 276and 277.Dv MRT6MSG_* 278for IPv4 and IPv6 respectively. 279The values of the rest of the upcall header fields 280and the body of the upcall message depend on the particular upcall type. 281.Pp 282If the upcall message type is 283.Dv IGMPMSG_NOCACHE 284or 285.Dv MRT6MSG_NOCACHE , 286this is an indication that a multicast packet has reached the multicast 287router, but the router has no forwarding state for that packet. 288Typically, the upcall would be a signal for the multicast routing 289user-level process to install the appropriate Multicast Forwarding 290Cache (MFC) entry in the kernel. 291.Pp 292An MFC entry is added by: 293.Bd -literal 294/* IPv4 */ 295struct mfcctl mc; 296memset(\*[Am]mc, 0, sizeof(mc)); 297memcpy(\*[Am]mc.mfcc_origin, \*[Am]source_addr, sizeof(mc.mfcc_origin)); 298memcpy(\*[Am]mc.mfcc_mcastgrp, \*[Am]group_addr, sizeof(mc.mfcc_mcastgrp)); 299mc.mfcc_parent = iif_index; 300for (i = 0; i \*[Lt] maxvifs; i++) 301 mc.mfcc_ttls[i] = oifs_ttl[i]; 302setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_MFC, 303 (void *)\*[Am]mc, sizeof(mc)); 304.Ed 305.Bd -literal 306/* IPv6 */ 307struct mf6cctl mc; 308memset(\*[Am]mc, 0, sizeof(mc)); 309memcpy(\*[Am]mc.mf6cc_origin, \*[Am]source_addr, sizeof(mc.mf6cc_origin)); 310memcpy(\*[Am]mc.mf6cc_mcastgrp, \*[Am]group_addr, sizeof(mf6cc_mcastgrp)); 311mc.mf6cc_parent = iif_index; 312for (i = 0; i \*[Lt] maxvifs; i++) 313 if (oifs_ttl[i] \*[Gt] 0) 314 IF_SET(i, \*[Am]mc.mf6cc_ifset); 315setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_ADD_MFC, 316 (void *)\*[Am]mc, sizeof(mc)); 317.Ed 318.Pp 319The 320.Va source_addr 321and 322.Va group_addr 323are the source and group address of the multicast packet (as set 324in the upcall message). 325The 326.Va iif_index 327is the virtual interface index of the multicast interface the multicast 328packets for this specific source and group address should be received on. 329The 330.Va oifs_ttl[] 331array contains the minimum TTL (per interface) a multicast packet 332should have to be forwarded on an outgoing interface. 333If the TTL value is zero, the corresponding interface is not included 334in the set of outgoing interfaces. 335Note that in case of IPv6 only the set of outgoing interfaces can 336be specified. 337.Pp 338An MFC entry is deleted by: 339.Bd -literal 340/* IPv4 */ 341struct mfcctl mc; 342memset(\*[Am]mc, 0, sizeof(mc)); 343memcpy(\*[Am]mc.mfcc_origin, \*[Am]source_addr, sizeof(mc.mfcc_origin)); 344memcpy(\*[Am]mc.mfcc_mcastgrp, \*[Am]group_addr, sizeof(mc.mfcc_mcastgrp)); 345setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_MFC, 346 (void *)\*[Am]mc, sizeof(mc)); 347.Ed 348.Bd -literal 349/* IPv6 */ 350struct mf6cctl mc; 351memset(\*[Am]mc, 0, sizeof(mc)); 352memcpy(\*[Am]mc.mf6cc_origin, \*[Am]source_addr, sizeof(mc.mf6cc_origin)); 353memcpy(\*[Am]mc.mf6cc_mcastgrp, \*[Am]group_addr, sizeof(mf6cc_mcastgrp)); 354setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_DEL_MFC, 355 (void *)\*[Am]mc, sizeof(mc)); 356.Ed 357.Pp 358The following method can be used to get various statistics per 359installed MFC entry in the kernel (e.g., the number of forwarded 360packets per source and group address): 361.Bd -literal 362/* IPv4 */ 363struct sioc_sg_req sgreq; 364memset(\*[Am]sgreq, 0, sizeof(sgreq)); 365memcpy(\*[Am]sgreq.src, \*[Am]source_addr, sizeof(sgreq.src)); 366memcpy(\*[Am]sgreq.grp, \*[Am]group_addr, sizeof(sgreq.grp)); 367ioctl(mrouter_s4, SIOCGETSGCNT, \*[Am]sgreq); 368.Ed 369.Bd -literal 370/* IPv6 */ 371struct sioc_sg_req6 sgreq; 372memset(\*[Am]sgreq, 0, sizeof(sgreq)); 373memcpy(\*[Am]sgreq.src, \*[Am]source_addr, sizeof(sgreq.src)); 374memcpy(\*[Am]sgreq.grp, \*[Am]group_addr, sizeof(sgreq.grp)); 375ioctl(mrouter_s6, SIOCGETSGCNT_IN6, \*[Am]sgreq); 376.Ed 377.Pp 378The following method can be used to get various statistics per 379multicast virtual interface in the kernel (e.g., the number of forwarded 380packets per interface): 381.Bd -literal 382/* IPv4 */ 383struct sioc_vif_req vreq; 384memset(\*[Am]vreq, 0, sizeof(vreq)); 385vreq.vifi = vif_index; 386ioctl(mrouter_s4, SIOCGETVIFCNT, \*[Am]vreq); 387.Ed 388.Bd -literal 389/* IPv6 */ 390struct sioc_mif_req6 mreq; 391memset(\*[Am]mreq, 0, sizeof(mreq)); 392mreq.mifi = vif_index; 393ioctl(mrouter_s6, SIOCGETMIFCNT_IN6, \*[Am]mreq); 394.Ed 395.Ss Advanced Multicast API Programming Guide 396If we want to add new features in the kernel, it becomes difficult 397to preserve backward compatibility (binary and API), 398and at the same time to allow user-level processes to take advantage of 399the new features (if the kernel supports them). 400.Pp 401One of the mechanisms that allows us to preserve the backward 402compatibility is a sort of negotiation 403between the user-level process and the kernel: 404.Bl -enum 405.It 406The user-level process tries to enable in the kernel the set of new 407features (and the corresponding API) it would like to use. 408.It 409The kernel returns the (sub)set of features it knows about 410and is willing to be enabled. 411.It 412The user-level process uses only that set of features 413the kernel has agreed on. 414.El 415.\" 416.Pp 417To support backward compatibility, if the user-level process does not 418ask for any new features, the kernel defaults to the basic 419multicast API (see the 420.Sx Programming Guide 421section). 422.\" XXX: edit as appropriate after the advanced multicast API is 423.\" supported under IPv6 424Currently, the advanced multicast API exists only for IPv4; 425in the future there will be IPv6 support as well. 426.Pp 427Below is a summary of the expandable API solution. 428Note that all new options and structures are defined 429in 430.In netinet/ip_mroute.h 431and 432.In netinet6/ip6_mroute.h , 433unless stated otherwise. 434.Pp 435The user-level process uses new 436.Fn getsockopt Ns / Ns Fn setsockopt 437options to 438perform the API features negotiation with the kernel. 439This negotiation must be performed right after the multicast routing 440socket is open. 441The set of desired/allowed features is stored in a bitset 442(currently, in 443.Vt uint32_t ; 444i.e., maximum of 32 new features). 445The new 446.Fn getsockopt Ns / Ns Fn setsockopt 447options are 448.Dv MRT_API_SUPPORT 449and 450.Dv MRT_API_CONFIG . 451Example: 452.Bd -literal 453uint32_t v; 454getsockopt(sock, IPPROTO_IP, MRT_API_SUPPORT, (void *)\*[Am]v, sizeof(v)); 455.Ed 456.Pp 457would set in 458.Va v 459the pre-defined bits that the kernel API supports. 460The eight least significant bits in 461.Vt uint32_t 462are same as the 463eight possible flags 464.Dv MRT_MFC_FLAGS_* 465that can be used in 466.Va mfcc_flags 467as part of the new definition of 468.Vt "struct mfcctl" 469(see below about those flags), which leaves 24 flags for other new features. 470The value returned by 471.Fn getsockopt MRT_API_SUPPORT 472is read-only; in other words, 473.Fn setsockopt MRT_API_SUPPORT 474would fail. 475.Pp 476To modify the API, and to set some specific feature in the kernel, then: 477.Bd -literal 478uint32_t v = MRT_MFC_FLAGS_DISABLE_WRONGVIF; 479if (setsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)\*[Am]v, sizeof(v)) 480 != 0) { 481 return (ERROR); 482} 483if (v \*[Am] MRT_MFC_FLAGS_DISABLE_WRONGVIF) 484 return (OK); /* Success */ 485else 486 return (ERROR); 487.Ed 488.Pp 489In other words, when 490.Fn setsockopt MRT_API_CONFIG 491is called, the 492argument to it specifies the desired set of features to 493be enabled in the API and the kernel. 494The return value in 495.Va v 496is the actual (sub)set of features that were enabled in the kernel. 497To obtain later the same set of features that were enabled, then: 498.Bd -literal 499getsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)\*[Am]v, sizeof(v)); 500.Ed 501.Pp 502The set of enabled features is global. 503In other words, 504.Fn setsockopt MRT_API_CONFIG 505should be called right after 506.Fn setsockopt MRT_INIT . 507.Pp 508Currently, the following set of new features is defined: 509.Bd -literal 510#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 \*[Lt]\*[Lt] 0) /* disable WRONGVIF signals */ 511#define MRT_MFC_FLAGS_BORDER_VIF (1 \*[Lt]\*[Lt] 1) /* border vif */ 512#define MRT_MFC_RP (1 \*[Lt]\*[Lt] 8) /* enable RP address */ 513#define MRT_MFC_BW_UPCALL (1 \*[Lt]\*[Lt] 9) /* enable bw upcalls */ 514.Ed 515.\" .Pp 516.\" In the future there might be: 517.\" .Bd -literal 518.\" #define MRT_MFC_GROUP_SPECIFIC (1 \*[Lt]\*[Lt] 10) /* allow (*,G) MFC entries */ 519.\" .Ed 520.\" .Pp 521.\" to allow (*,G) MFC entries (i.e., group-specific entries) in the kernel. 522.\" For now this is left-out until it is clear whether 523.\" (*,G) MFC support is the preferred solution instead of something more generic 524.\" solution for example. 525.\" 526.\" 2. The newly defined struct mfcctl2. 527.\" 528.Pp 529The advanced multicast API uses a newly defined 530.Vt "struct mfcctl2" 531instead of the traditional 532.Vt "struct mfcctl" . 533The original 534.Vt "struct mfcctl" 535is kept as is. 536The new 537.Vt "struct mfcctl2" 538is: 539.Bd -literal 540/* 541 * The new argument structure for MRT_ADD_MFC and MRT_DEL_MFC overlays 542 * and extends the old struct mfcctl. 543 */ 544struct mfcctl2 { 545 /* the mfcctl fields */ 546 struct in_addr mfcc_origin; /* ip origin of mcasts */ 547 struct in_addr mfcc_mcastgrp; /* multicast group associated*/ 548 vifi_t mfcc_parent; /* incoming vif */ 549 u_char mfcc_ttls[MAXVIFS];/* forwarding ttls on vifs */ 550 551 /* extension fields */ 552 uint8_t mfcc_flags[MAXVIFS];/* the MRT_MFC_FLAGS_* flags*/ 553 struct in_addr mfcc_rp; /* the RP address */ 554}; 555.Ed 556.Pp 557The new fields are 558.Va mfcc_flags[MAXVIFS] 559and 560.Va mfcc_rp . 561Note that for compatibility reasons they are added at the end. 562.Pp 563The 564.Va mfcc_flags[MAXVIFS] 565field is used to set various flags per 566interface per (S,G) entry. 567Currently, the defined flags are: 568.Bd -literal 569#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 \*[Lt]\*[Lt] 0) /* disable WRONGVIF signals */ 570#define MRT_MFC_FLAGS_BORDER_VIF (1 \*[Lt]\*[Lt] 1) /* border vif */ 571.Ed 572.Pp 573The 574.Dv MRT_MFC_FLAGS_DISABLE_WRONGVIF 575flag is used to explicitly disable the 576.Dv IGMPMSG_WRONGVIF 577kernel signal at the (S,G) granularity if a multicast data packet 578arrives on the wrong interface. 579Usually, this signal is used to 580complete the shortest-path switch in case of PIM-SM multicast routing, 581or to trigger a PIM assert message. 582However, it should not be delivered for interfaces that are not in 583the outgoing interface set, and that are not expecting to 584become an incoming interface. 585Hence, if the 586.Dv MRT_MFC_FLAGS_DISABLE_WRONGVIF 587flag is set for some of the 588interfaces, then a data packet that arrives on that interface for 589that MFC entry will NOT trigger a WRONGVIF signal. 590If that flag is not set, then a signal is triggered (the default action). 591.Pp 592The 593.Dv MRT_MFC_FLAGS_BORDER_VIF 594flag is used to specify whether the Border-bit in PIM 595Register messages should be set (in case when the Register encapsulation 596is performed inside the kernel). 597If it is set for the special PIM Register kernel virtual interface 598(see 599.Xr pim 4 ) , 600the Border-bit in the Register messages sent to the RP will be set. 601.Pp 602The remaining six bits are reserved for future usage. 603.Pp 604The 605.Va mfcc_rp 606field is used to specify the RP address (in case of PIM-SM multicast routing) 607for a multicast 608group G if we want to perform kernel-level PIM Register encapsulation. 609The 610.Va mfcc_rp 611field is used only if the 612.Dv MRT_MFC_RP 613advanced API flag/capability has been successfully set by 614.Fn setsockopt MRT_API_CONFIG . 615.Pp 616.\" 617.\" 3. Kernel-level PIM Register encapsulation 618.\" 619If the 620.Dv MRT_MFC_RP 621flag was successfully set by 622.Fn setsockopt MRT_API_CONFIG , 623then the kernel will attempt to perform 624the PIM Register encapsulation itself instead of sending the 625multicast data packets to user level (inside 626.Dv IGMPMSG_WHOLEPKT 627upcalls) for user-level encapsulation. 628The RP address would be taken from the 629.Va mfcc_rp 630field 631inside the new 632.Vt "struct mfcctl2" . 633However, even if the 634.Dv MRT_MFC_RP 635flag was successfully set, if the 636.Va mfcc_rp 637field was set to 638.Dv INADDR_ANY , 639then the 640kernel will still deliver an 641.Dv IGMPMSG_WHOLEPKT 642upcall with the 643multicast data packet to the user-level process. 644.Pp 645In addition, if the multicast data packet is too large to fit within 646a single IP packet after the PIM Register encapsulation (e.g., if 647its size was on the order of 65500 bytes), the data packet will be 648fragmented, and then each of the fragments will be encapsulated 649separately. 650Note that typically a multicast data packet can be that 651large only if it was originated locally from the same hosts that 652performs the encapsulation; otherwise the transmission of the 653multicast data packet over Ethernet for example would have 654fragmented it into much smaller pieces. 655.\" 656.\" Note that if this code is ported to IPv6, we may need the kernel to 657.\" perform MTU discovery to the RP, and keep those discoveries inside 658.\" the kernel so the encapsulating router may send back ICMP 659.\" Fragmentation Required if the size of the multicast data packet is 660.\" too large (see "Encapsulating data packets in the Register Tunnel" 661.\" in Section 4.4.1 in the PIM-SM spec 662.\" draft-ietf-pim-sm-v2-new-05.{txt,ps}). 663.\" For IPv4 we may be able to get away without it, but for IPv6 we need 664.\" that. 665.\" 666.\" 4. Mechanism for "multicast bandwidth monitoring and upcalls". 667.\" 668.Pp 669Typically, a multicast routing user-level process would need to know the 670forwarding bandwidth for some data flow. 671For example, the multicast routing process may want to timeout idle MFC 672entries, or in case of PIM-SM it can initiate (S,G) shortest-path switch if 673the bandwidth rate is above a threshold for example. 674.Pp 675The original solution for measuring the bandwidth of a dataflow was 676that a user-level process would periodically 677query the kernel about the number of forwarded packets/bytes per 678(S,G), and then based on those numbers it would estimate whether a source 679has been idle, or whether the source's transmission bandwidth is above a 680threshold. 681That solution is far from being scalable, hence the need for a new 682mechanism for bandwidth monitoring. 683.Pp 684Below is a description of the bandwidth monitoring mechanism. 685.Bl -bullet 686.It 687If the bandwidth of a data flow satisfies some pre-defined filter, 688the kernel delivers an upcall on the multicast routing socket 689to the multicast routing process that has installed that filter. 690.It 691The bandwidth-upcall filters are installed per (S,G). 692There can be 693more than one filter per (S,G). 694.It 695Instead of supporting all possible comparison operations 696(i.e., \*[Lt] \*[Lt]= == != \*[Gt] \*[Gt]= ), there is support only for the 697\*[Lt]= and \*[Gt]= operations, 698because this makes the kernel-level implementation simpler, 699and because practically we need only those two. 700Further, the missing operations can be simulated by secondary 701user-level filtering of those \*[Lt]= and \*[Gt]= filters. 702For example, to simulate !=, then we need to install filter 703.Dq bw \*[Lt]= 0xffffffff , 704and after an 705upcall is received, we need to check whether 706.Dq measured_bw != expected_bw . 707.It 708The bandwidth-upcall mechanism is enabled by 709.Fn setsockopt MRT_API_CONFIG 710for the 711.Dv MRT_MFC_BW_UPCALL 712flag. 713.It 714The bandwidth-upcall filters are added/deleted by the new 715.Fn setsockopt MRT_ADD_BW_UPCALL 716and 717.Fn setsockopt MRT_DEL_BW_UPCALL 718respectively (with the appropriate 719.Vt "struct bw_upcall" 720argument of course). 721.El 722.Pp 723From application point of view, a developer needs to know about 724the following: 725.Bd -literal 726/* 727 * Structure for installing or delivering an upcall if the 728 * measured bandwidth is above or below a threshold. 729 * 730 * User programs (e.g. daemons) may have a need to know when the 731 * bandwidth used by some data flow is above or below some threshold. 732 * This interface allows the userland to specify the threshold (in 733 * bytes and/or packets) and the measurement interval. Flows are 734 * all packet with the same source and destination IP address. 735 * At the moment the code is only used for multicast destinations 736 * but there is nothing that prevents its use for unicast. 737 * 738 * The measurement interval cannot be shorter than some Tmin (currently, 3s). 739 * The threshold is set in packets and/or bytes per_interval. 740 * 741 * Measurement works as follows: 742 * 743 * For \*[Gt]= measurements: 744 * The first packet marks the start of a measurement interval. 745 * During an interval we count packets and bytes, and when we 746 * pass the threshold we deliver an upcall and we are done. 747 * The first packet after the end of the interval resets the 748 * count and restarts the measurement. 749 * 750 * For \*[Lt]= measurement: 751 * We start a timer to fire at the end of the interval, and 752 * then for each incoming packet we count packets and bytes. 753 * When the timer fires, we compare the value with the threshold, 754 * schedule an upcall if we are below, and restart the measurement 755 * (reschedule timer and zero counters). 756 */ 757 758struct bw_data { 759 struct timeval b_time; 760 uint64_t b_packets; 761 uint64_t b_bytes; 762}; 763 764struct bw_upcall { 765 struct in_addr bu_src; /* source address */ 766 struct in_addr bu_dst; /* destination address */ 767 uint32_t bu_flags; /* misc flags (see below) */ 768#define BW_UPCALL_UNIT_PACKETS (1 \*[Lt]\*[Lt] 0) /* threshold (in packets) */ 769#define BW_UPCALL_UNIT_BYTES (1 \*[Lt]\*[Lt] 1) /* threshold (in bytes) */ 770#define BW_UPCALL_GEQ (1 \*[Lt]\*[Lt] 2) /* upcall if bw \*[Gt]= threshold */ 771#define BW_UPCALL_LEQ (1 \*[Lt]\*[Lt] 3) /* upcall if bw \*[Lt]= threshold */ 772#define BW_UPCALL_DELETE_ALL (1 \*[Lt]\*[Lt] 4) /* delete all upcalls for s,d*/ 773 struct bw_data bu_threshold; /* the bw threshold */ 774 struct bw_data bu_measured; /* the measured bw */ 775}; 776 777/* max. number of upcalls to deliver together */ 778#define BW_UPCALLS_MAX 128 779/* min. threshold time interval for bandwidth measurement */ 780#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_SEC 3 781#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_USEC 0 782.Ed 783.Pp 784The 785.Vt bw_upcall 786structure is used as an argument to 787.Fn setsockopt MRT_ADD_BW_UPCALL 788and 789.Fn setsockopt MRT_DEL_BW_UPCALL . 790Each 791.Fn setsockopt MRT_ADD_BW_UPCALL 792installs a filter in the kernel 793for the source and destination address in the 794.Vt bw_upcall 795argument, 796and that filter will trigger an upcall according to the following 797pseudo-algorithm: 798.Bd -literal 799 if (bw_upcall_oper IS "\*[Gt]=") { 800 if (((bw_upcall_unit \*[Am] PACKETS == PACKETS) \*[Am]\*[Am] 801 (measured_packets \*[Gt]= threshold_packets)) || 802 ((bw_upcall_unit \*[Am] BYTES == BYTES) \*[Am]\*[Am] 803 (measured_bytes \*[Gt]= threshold_bytes))) 804 SEND_UPCALL("measured bandwidth is \*[Gt]= threshold"); 805 } 806 if (bw_upcall_oper IS "\*[Lt]=" \*[Am]\*[Am] measured_interval \*[Gt]= threshold_interval) { 807 if (((bw_upcall_unit \*[Am] PACKETS == PACKETS) \*[Am]\*[Am] 808 (measured_packets \*[Lt]= threshold_packets)) || 809 ((bw_upcall_unit \*[Am] BYTES == BYTES) \*[Am]\*[Am] 810 (measured_bytes \*[Lt]= threshold_bytes))) 811 SEND_UPCALL("measured bandwidth is \*[Lt]= threshold"); 812 } 813.Ed 814.Pp 815In the same 816.Vt bw_upcall 817the unit can be specified in both BYTES and PACKETS. 818However, the GEQ and LEQ flags are mutually exclusive. 819.Pp 820Basically, an upcall is delivered if the measured bandwidth is \*[Gt]= or 821\*[Lt]= the threshold bandwidth (within the specified measurement 822interval). 823For practical reasons, the smallest value for the measurement 824interval is 3 seconds. 825If smaller values are allowed, then the bandwidth 826estimation may be less accurate, or the potentially very high frequency 827of the generated upcalls may introduce too much overhead. 828For the \*[Gt]= operation, the answer may be known before the end of 829.Va threshold_interval , 830therefore the upcall may be delivered earlier. 831For the \*[Lt]= operation however, we must wait 832until the threshold interval has expired to know the answer. 833.Pp 834Example of usage: 835.Bd -literal 836struct bw_upcall bw_upcall; 837/* Assign all bw_upcall fields as appropriate */ 838memset(\*[Am]bw_upcall, 0, sizeof(bw_upcall)); 839memcpy(\*[Am]bw_upcall.bu_src, \*[Am]source, sizeof(bw_upcall.bu_src)); 840memcpy(\*[Am]bw_upcall.bu_dst, \*[Am]group, sizeof(bw_upcall.bu_dst)); 841bw_upcall.bu_threshold.b_data = threshold_interval; 842bw_upcall.bu_threshold.b_packets = threshold_packets; 843bw_upcall.bu_threshold.b_bytes = threshold_bytes; 844if (is_threshold_in_packets) 845 bw_upcall.bu_flags |= BW_UPCALL_UNIT_PACKETS; 846if (is_threshold_in_bytes) 847 bw_upcall.bu_flags |= BW_UPCALL_UNIT_BYTES; 848do { 849 if (is_geq_upcall) { 850 bw_upcall.bu_flags |= BW_UPCALL_GEQ; 851 break; 852 } 853 if (is_leq_upcall) { 854 bw_upcall.bu_flags |= BW_UPCALL_LEQ; 855 break; 856 } 857 return (ERROR); 858} while (0); 859setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_BW_UPCALL, 860 (void *)\*[Am]bw_upcall, sizeof(bw_upcall)); 861.Ed 862.Pp 863To delete a single filter, then use 864.Dv MRT_DEL_BW_UPCALL , 865and the fields of bw_upcall must be set 866exactly same as when 867.Dv MRT_ADD_BW_UPCALL 868was called. 869.Pp 870To delete all bandwidth filters for a given (S,G), then 871only the 872.Va bu_src 873and 874.Va bu_dst 875fields in 876.Vt "struct bw_upcall" 877need to be set, and then just set only the 878.Dv BW_UPCALL_DELETE_ALL 879flag inside field 880.Va bw_upcall.bu_flags . 881.Pp 882The bandwidth upcalls are received by aggregating them in the new upcall 883message: 884.Bd -literal 885#define IGMPMSG_BW_UPCALL 4 /* BW monitoring upcall */ 886.Ed 887.Pp 888This message is an array of 889.Vt "struct bw_upcall" 890elements (up to 891.Dv BW_UPCALLS_MAX 892= 128). 893The upcalls are 894delivered when there are 128 pending upcalls, or when 1 second has 895expired since the previous upcall (whichever comes first). 896In an 897.Vt "struct upcall" 898element, the 899.Va bu_measured 900field is filled-in to 901indicate the particular measured values. 902However, because of the way 903the particular intervals are measured, the user should be careful how 904.Va bu_measured.b_time 905is used. 906For example, if the 907filter is installed to trigger an upcall if the number of packets 908is \*[Gt]= 1, then 909.Va bu_measured 910may have a value of zero in the upcalls after the 911first one, because the measured interval for \*[Gt]= filters is 912.Dq clocked 913by the forwarded packets. 914Hence, this upcall mechanism should not be used for measuring 915the exact value of the bandwidth of the forwarded data. 916To measure the exact bandwidth, the user would need to 917get the forwarded packets statistics with the 918.Fn ioctl SIOCGETSGCNT 919mechanism 920(see the 921.Sx Programming Guide 922section) . 923.Pp 924Note that the upcalls for a filter are delivered until the specific 925filter is deleted, but no more frequently than once per 926.Va bu_threshold.b_time . 927For example, if the filter is specified to 928deliver a signal if bw \*[Gt]= 1 packet, the first packet will trigger a 929signal, but the next upcall will be triggered no earlier than 930.Va bu_threshold.b_time 931after the previous upcall. 932.\" 933.Sh SEE ALSO 934.Xr getsockopt 2 , 935.Xr recvfrom 2 , 936.Xr recvmsg 2 , 937.Xr setsockopt 2 , 938.Xr socket 2 , 939.Xr icmp6 4 , 940.Xr inet 4 , 941.Xr inet6 4 , 942.Xr intro 4 , 943.Xr ip 4 , 944.Xr ip6 4 , 945.Xr pim 4 946.\" 947.Sh AUTHORS 948.An -nosplit 949The original multicast code was written by 950.An David Waitzman 951(BBN Labs), 952and later modified by the following individuals: 953.An Steve Deering 954(Stanford), 955.An Mark J. Steiglitz 956(Stanford), 957.An Van Jacobson 958(LBL), 959.An Ajit Thyagarajan 960(PARC), 961.An Bill Fenner 962(PARC). 963The IPv6 multicast support was implemented by the KAME project 964.Pq Pa http://www.kame.net , 965and was based on the IPv4 multicast code. 966The advanced multicast API and the multicast bandwidth 967monitoring were implemented by 968.An Pavlin Radoslavov 969(ICSI) 970in collaboration with 971.An Chris Brown 972(NextHop). 973.Pp 974This manual page was written by 975.An Pavlin Radoslavov 976(ICSI). 977