1.\" Copyright (c) 2001-2003 International Computer Science Institute 2.\" 3.\" Permission is hereby granted, free of charge, to any person obtaining a 4.\" copy of this software and associated documentation files (the "Software"), 5.\" to deal in the Software without restriction, including without limitation 6.\" the rights to use, copy, modify, merge, publish, distribute, sublicense, 7.\" and/or sell copies of the Software, and to permit persons to whom the 8.\" Software is furnished to do so, subject to the following conditions: 9.\" 10.\" The above copyright notice and this permission notice shall be included in 11.\" all copies or substantial portions of the Software. 12.\" 13.\" The names and trademarks of copyright holders may not be used in 14.\" advertising or publicity pertaining to the software without specific 15.\" prior permission. Title to copyright in this software and any associated 16.\" documentation will at all times remain with the copyright holders. 17.\" 18.\" THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19.\" IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20.\" FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21.\" AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22.\" LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 23.\" FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 24.\" DEALINGS IN THE SOFTWARE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd February 25, 2007 29.Dt MULTICAST 4 30.Os 31.\" 32.Sh NAME 33.Nm multicast 34.Nd Multicast Routing 35.\" 36.Sh SYNOPSIS 37.Cd "options MROUTING" 38.Pp 39.In sys/types.h 40.In sys/socket.h 41.In netinet/in.h 42.In netinet/ip_mroute.h 43.In netinet6/ip6_mroute.h 44.Ft int 45.Fn getsockopt "int s" IPPROTO_IP MRT_INIT "void *optval" "socklen_t *optlen" 46.Ft int 47.Fn setsockopt "int s" IPPROTO_IP MRT_INIT "const void *optval" "socklen_t optlen" 48.Ft int 49.Fn getsockopt "int s" IPPROTO_IPV6 MRT6_INIT "void *optval" "socklen_t *optlen" 50.Ft int 51.Fn setsockopt "int s" IPPROTO_IPV6 MRT6_INIT "const void *optval" "socklen_t optlen" 52.Sh DESCRIPTION 53.Tn "Multicast routing" 54is used to efficiently propagate data 55packets to a set of multicast listeners in multipoint networks. 56If unicast is used to replicate the data to all listeners, 57then some of the network links may carry multiple copies of the same 58data packets. 59With multicast routing, the overhead is reduced to one copy 60(at most) per network link. 61.Pp 62All multicast-capable routers must run a common multicast routing 63protocol. 64The Distance Vector Multicast Routing Protocol (DVMRP) 65was the first developed multicast routing protocol. 66Later, other protocols such as Multicast Extensions to OSPF (MOSPF), 67Core Based Trees (CBT), 68Protocol Independent Multicast - Sparse Mode (PIM-SM), 69and Protocol Independent Multicast - Dense Mode (PIM-DM) 70were developed as well. 71.Pp 72To start multicast routing, 73the user must enable multicast forwarding in the kernel 74(see 75.Sx SYNOPSIS 76about the kernel configuration options), 77and must run a multicast routing capable user-level process. 78From developer's point of view, 79the programming guide described in the 80.Sx "Programming Guide" 81section should be used to control the multicast forwarding in the kernel. 82.\" 83.Ss Programming Guide 84This section provides information about the basic multicast routing API. 85The so-called 86.Dq advanced multicast API 87is described in the 88.Sx "Advanced Multicast API Programming Guide" 89section. 90.Pp 91First, a multicast routing socket must be open. 92That socket would be used 93to control the multicast forwarding in the kernel. 94Note that most operations below require certain privilege 95(i.e., root privilege): 96.Bd -literal 97/* IPv4 */ 98int mrouter_s4; 99mrouter_s4 = socket(AF_INET, SOCK_RAW, IPPROTO_IGMP); 100.Ed 101.Bd -literal 102int mrouter_s6; 103mrouter_s6 = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); 104.Ed 105.Pp 106Note that if the router needs to open an IGMP or ICMPv6 socket 107(in case of IPv4 and IPv6 respectively) 108for sending or receiving of IGMP or MLD multicast group membership messages, 109then the same 110.Va mrouter_s4 111or 112.Va mrouter_s6 113sockets should be used 114for sending and receiving respectively IGMP or MLD messages. 115In case of 116.Bx Ns 117-derived kernel, it may be possible to open separate sockets 118for IGMP or MLD messages only. 119However, some other kernels (e.g., 120.Tn Linux ) 121require that the multicast 122routing socket must be used for sending and receiving of IGMP or MLD 123messages. 124Therefore, for portability reason the multicast 125routing socket should be reused for IGMP and MLD messages as well. 126.Pp 127After the multicast routing socket is open, it can be used to enable 128or disable multicast forwarding in the kernel: 129.Bd -literal 130/* IPv4 */ 131int v = 1; /* 1 to enable, or 0 to disable */ 132setsockopt(mrouter_s4, IPPROTO_IP, MRT_INIT, (void *)&v, sizeof(v)); 133.Ed 134.Bd -literal 135/* IPv6 */ 136int v = 1; /* 1 to enable, or 0 to disable */ 137setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_INIT, (void *)&v, sizeof(v)); 138\&... 139/* If necessary, filter all ICMPv6 messages */ 140struct icmp6_filter filter; 141ICMP6_FILTER_SETBLOCKALL(&filter); 142setsockopt(mrouter_s6, IPPROTO_ICMPV6, ICMP6_FILTER, (void *)&filter, 143 sizeof(filter)); 144.Ed 145.Pp 146After multicast forwarding is enabled, the multicast routing socket 147can be used to enable PIM processing in the kernel if we are running PIM-SM or 148PIM-DM 149(see 150.Xr pim 4 ) . 151.Pp 152For each network interface (e.g., physical or a virtual tunnel) 153that would be used for multicast forwarding, a corresponding 154multicast interface must be added to the kernel: 155.Bd -literal 156/* IPv4 */ 157struct vifctl vc; 158memset(&vc, 0, sizeof(vc)); 159/* Assign all vifctl fields as appropriate */ 160vc.vifc_vifi = vif_index; 161vc.vifc_flags = vif_flags; 162vc.vifc_threshold = min_ttl_threshold; 163vc.vifc_rate_limit = 0; 164memcpy(&vc.vifc_lcl_addr, &vif_local_address, sizeof(vc.vifc_lcl_addr)); 165setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_VIF, (void *)&vc, 166 sizeof(vc)); 167.Ed 168.Pp 169The 170.Va vif_index 171must be unique per vif. 172The 173.Va vif_flags 174contains the 175.Dv VIFF_* 176flags as defined in 177.In netinet/ip_mroute.h . 178The 179.Dv VIFF_TUNNEL 180flag is no longer supported by 181.Fx . 182Users who wish to forward multicast datagrams over a tunnel should consider 183configuring a 184.Xr gif 4 185or 186.Xr gre 4 187tunnel and using it as a physical interface. 188.Pp 189The 190.Va min_ttl_threshold 191contains the minimum TTL a multicast data packet must have to be 192forwarded on that vif. 193Typically, it would have value of 1. 194.Pp 195The 196.Va max_rate_limit 197argument is no longer supported in 198.Fx 199and should be set to 0. 200Users who wish to rate-limit multicast datagrams should consider the use of 201.Xr dummynet 4 202or 203.Xr altq 4 . 204.Pp 205The 206.Va vif_local_address 207contains the local IP address of the corresponding local interface. 208The 209.Va vif_remote_address 210contains the remote IP address in case of DVMRP multicast tunnels. 211.Bd -literal 212/* IPv6 */ 213struct mif6ctl mc; 214memset(&mc, 0, sizeof(mc)); 215/* Assign all mif6ctl fields as appropriate */ 216mc.mif6c_mifi = mif_index; 217mc.mif6c_flags = mif_flags; 218mc.mif6c_pifi = pif_index; 219setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_ADD_MIF, (void *)&mc, 220 sizeof(mc)); 221.Ed 222.Pp 223The 224.Va mif_index 225must be unique per vif. 226The 227.Va mif_flags 228contains the 229.Dv MIFF_* 230flags as defined in 231.In netinet6/ip6_mroute.h . 232The 233.Va pif_index 234is the physical interface index of the corresponding local interface. 235.Pp 236A multicast interface is deleted by: 237.Bd -literal 238/* IPv4 */ 239vifi_t vifi = vif_index; 240setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_VIF, (void *)&vifi, 241 sizeof(vifi)); 242.Ed 243.Bd -literal 244/* IPv6 */ 245mifi_t mifi = mif_index; 246setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_DEL_MIF, (void *)&mifi, 247 sizeof(mifi)); 248.Ed 249.Pp 250After the multicast forwarding is enabled, and the multicast virtual 251interfaces are 252added, the kernel may deliver upcall messages (also called signals 253later in this text) on the multicast routing socket that was open 254earlier with 255.Dv MRT_INIT 256or 257.Dv MRT6_INIT . 258The IPv4 upcalls have 259.Vt "struct igmpmsg" 260header (see 261.In netinet/ip_mroute.h ) 262with field 263.Va im_mbz 264set to zero. 265Note that this header follows the structure of 266.Vt "struct ip" 267with the protocol field 268.Va ip_p 269set to zero. 270The IPv6 upcalls have 271.Vt "struct mrt6msg" 272header (see 273.In netinet6/ip6_mroute.h ) 274with field 275.Va im6_mbz 276set to zero. 277Note that this header follows the structure of 278.Vt "struct ip6_hdr" 279with the next header field 280.Va ip6_nxt 281set to zero. 282.Pp 283The upcall header contains field 284.Va im_msgtype 285and 286.Va im6_msgtype 287with the type of the upcall 288.Dv IGMPMSG_* 289and 290.Dv MRT6MSG_* 291for IPv4 and IPv6 respectively. 292The values of the rest of the upcall header fields 293and the body of the upcall message depend on the particular upcall type. 294.Pp 295If the upcall message type is 296.Dv IGMPMSG_NOCACHE 297or 298.Dv MRT6MSG_NOCACHE , 299this is an indication that a multicast packet has reached the multicast 300router, but the router has no forwarding state for that packet. 301Typically, the upcall would be a signal for the multicast routing 302user-level process to install the appropriate Multicast Forwarding 303Cache (MFC) entry in the kernel. 304.Pp 305An MFC entry is added by: 306.Bd -literal 307/* IPv4 */ 308struct mfcctl mc; 309memset(&mc, 0, sizeof(mc)); 310memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 311memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 312mc.mfcc_parent = iif_index; 313for (i = 0; i < maxvifs; i++) 314 mc.mfcc_ttls[i] = oifs_ttl[i]; 315setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_MFC, 316 (void *)&mc, sizeof(mc)); 317.Ed 318.Bd -literal 319/* IPv6 */ 320struct mf6cctl mc; 321memset(&mc, 0, sizeof(mc)); 322memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 323memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 324mc.mf6cc_parent = iif_index; 325for (i = 0; i < maxvifs; i++) 326 if (oifs_ttl[i] > 0) 327 IF_SET(i, &mc.mf6cc_ifset); 328setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_ADD_MFC, 329 (void *)&mc, sizeof(mc)); 330.Ed 331.Pp 332The 333.Va source_addr 334and 335.Va group_addr 336are the source and group address of the multicast packet (as set 337in the upcall message). 338The 339.Va iif_index 340is the virtual interface index of the multicast interface the multicast 341packets for this specific source and group address should be received on. 342The 343.Va oifs_ttl[] 344array contains the minimum TTL (per interface) a multicast packet 345should have to be forwarded on an outgoing interface. 346If the TTL value is zero, the corresponding interface is not included 347in the set of outgoing interfaces. 348Note that in case of IPv6 only the set of outgoing interfaces can 349be specified. 350.Pp 351An MFC entry is deleted by: 352.Bd -literal 353/* IPv4 */ 354struct mfcctl mc; 355memset(&mc, 0, sizeof(mc)); 356memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 357memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 358setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_MFC, 359 (void *)&mc, sizeof(mc)); 360.Ed 361.Bd -literal 362/* IPv6 */ 363struct mf6cctl mc; 364memset(&mc, 0, sizeof(mc)); 365memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 366memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 367setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_DEL_MFC, 368 (void *)&mc, sizeof(mc)); 369.Ed 370.Pp 371The following method can be used to get various statistics per 372installed MFC entry in the kernel (e.g., the number of forwarded 373packets per source and group address): 374.Bd -literal 375/* IPv4 */ 376struct sioc_sg_req sgreq; 377memset(&sgreq, 0, sizeof(sgreq)); 378memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 379memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 380ioctl(mrouter_s4, SIOCGETSGCNT, &sgreq); 381.Ed 382.Bd -literal 383/* IPv6 */ 384struct sioc_sg_req6 sgreq; 385memset(&sgreq, 0, sizeof(sgreq)); 386memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 387memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 388ioctl(mrouter_s6, SIOCGETSGCNT_IN6, &sgreq); 389.Ed 390.Pp 391The following method can be used to get various statistics per 392multicast virtual interface in the kernel (e.g., the number of forwarded 393packets per interface): 394.Bd -literal 395/* IPv4 */ 396struct sioc_vif_req vreq; 397memset(&vreq, 0, sizeof(vreq)); 398vreq.vifi = vif_index; 399ioctl(mrouter_s4, SIOCGETVIFCNT, &vreq); 400.Ed 401.Bd -literal 402/* IPv6 */ 403struct sioc_mif_req6 mreq; 404memset(&mreq, 0, sizeof(mreq)); 405mreq.mifi = vif_index; 406ioctl(mrouter_s6, SIOCGETMIFCNT_IN6, &mreq); 407.Ed 408.Ss Advanced Multicast API Programming Guide 409If we want to add new features in the kernel, it becomes difficult 410to preserve backward compatibility (binary and API), 411and at the same time to allow user-level processes to take advantage of 412the new features (if the kernel supports them). 413.Pp 414One of the mechanisms that allows us to preserve the backward 415compatibility is a sort of negotiation 416between the user-level process and the kernel: 417.Bl -enum 418.It 419The user-level process tries to enable in the kernel the set of new 420features (and the corresponding API) it would like to use. 421.It 422The kernel returns the (sub)set of features it knows about 423and is willing to be enabled. 424.It 425The user-level process uses only that set of features 426the kernel has agreed on. 427.El 428.\" 429.Pp 430To support backward compatibility, if the user-level process does not 431ask for any new features, the kernel defaults to the basic 432multicast API (see the 433.Sx "Programming Guide" 434section). 435.\" XXX: edit as appropriate after the advanced multicast API is 436.\" supported under IPv6 437Currently, the advanced multicast API exists only for IPv4; 438in the future there will be IPv6 support as well. 439.Pp 440Below is a summary of the expandable API solution. 441Note that all new options and structures are defined 442in 443.In netinet/ip_mroute.h 444and 445.In netinet6/ip6_mroute.h , 446unless stated otherwise. 447.Pp 448The user-level process uses new 449.Fn getsockopt Ns / Ns Fn setsockopt 450options to 451perform the API features negotiation with the kernel. 452This negotiation must be performed right after the multicast routing 453socket is open. 454The set of desired/allowed features is stored in a bitset 455(currently, in 456.Vt uint32_t ; 457i.e., maximum of 32 new features). 458The new 459.Fn getsockopt Ns / Ns Fn setsockopt 460options are 461.Dv MRT_API_SUPPORT 462and 463.Dv MRT_API_CONFIG . 464Example: 465.Bd -literal 466uint32_t v; 467getsockopt(sock, IPPROTO_IP, MRT_API_SUPPORT, (void *)&v, sizeof(v)); 468.Ed 469.Pp 470would set in 471.Va v 472the pre-defined bits that the kernel API supports. 473The eight least significant bits in 474.Vt uint32_t 475are same as the 476eight possible flags 477.Dv MRT_MFC_FLAGS_* 478that can be used in 479.Va mfcc_flags 480as part of the new definition of 481.Vt "struct mfcctl" 482(see below about those flags), which leaves 24 flags for other new features. 483The value returned by 484.Fn getsockopt MRT_API_SUPPORT 485is read-only; in other words, 486.Fn setsockopt MRT_API_SUPPORT 487would fail. 488.Pp 489To modify the API, and to set some specific feature in the kernel, then: 490.Bd -literal 491uint32_t v = MRT_MFC_FLAGS_DISABLE_WRONGVIF; 492if (setsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)) 493 != 0) { 494 return (ERROR); 495} 496if (v & MRT_MFC_FLAGS_DISABLE_WRONGVIF) 497 return (OK); /* Success */ 498else 499 return (ERROR); 500.Ed 501.Pp 502In other words, when 503.Fn setsockopt MRT_API_CONFIG 504is called, the 505argument to it specifies the desired set of features to 506be enabled in the API and the kernel. 507The return value in 508.Va v 509is the actual (sub)set of features that were enabled in the kernel. 510To obtain later the same set of features that were enabled, then: 511.Bd -literal 512getsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)); 513.Ed 514.Pp 515The set of enabled features is global. 516In other words, 517.Fn setsockopt MRT_API_CONFIG 518should be called right after 519.Fn setsockopt MRT_INIT . 520.Pp 521Currently, the following set of new features is defined: 522.Bd -literal 523#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 524#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 525#define MRT_MFC_RP (1 << 8) /* enable RP address */ 526#define MRT_MFC_BW_UPCALL (1 << 9) /* enable bw upcalls */ 527.Ed 528.\" .Pp 529.\" In the future there might be: 530.\" .Bd -literal 531.\" #define MRT_MFC_GROUP_SPECIFIC (1 << 10) /* allow (*,G) MFC entries */ 532.\" .Ed 533.\" .Pp 534.\" to allow (*,G) MFC entries (i.e., group-specific entries) in the kernel. 535.\" For now this is left-out until it is clear whether 536.\" (*,G) MFC support is the preferred solution instead of something more generic 537.\" solution for example. 538.\" 539.\" 2. The newly defined struct mfcctl2. 540.\" 541.Pp 542The advanced multicast API uses a newly defined 543.Vt "struct mfcctl2" 544instead of the traditional 545.Vt "struct mfcctl" . 546The original 547.Vt "struct mfcctl" 548is kept as is. 549The new 550.Vt "struct mfcctl2" 551is: 552.Bd -literal 553/* 554 * The new argument structure for MRT_ADD_MFC and MRT_DEL_MFC overlays 555 * and extends the old struct mfcctl. 556 */ 557struct mfcctl2 { 558 /* the mfcctl fields */ 559 struct in_addr mfcc_origin; /* ip origin of mcasts */ 560 struct in_addr mfcc_mcastgrp; /* multicast group associated*/ 561 vifi_t mfcc_parent; /* incoming vif */ 562 u_char mfcc_ttls[MAXVIFS];/* forwarding ttls on vifs */ 563 564 /* extension fields */ 565 uint8_t mfcc_flags[MAXVIFS];/* the MRT_MFC_FLAGS_* flags*/ 566 struct in_addr mfcc_rp; /* the RP address */ 567}; 568.Ed 569.Pp 570The new fields are 571.Va mfcc_flags[MAXVIFS] 572and 573.Va mfcc_rp . 574Note that for compatibility reasons they are added at the end. 575.Pp 576The 577.Va mfcc_flags[MAXVIFS] 578field is used to set various flags per 579interface per (S,G) entry. 580Currently, the defined flags are: 581.Bd -literal 582#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 583#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 584.Ed 585.Pp 586The 587.Dv MRT_MFC_FLAGS_DISABLE_WRONGVIF 588flag is used to explicitly disable the 589.Dv IGMPMSG_WRONGVIF 590kernel signal at the (S,G) granularity if a multicast data packet 591arrives on the wrong interface. 592Usually, this signal is used to 593complete the shortest-path switch in case of PIM-SM multicast routing, 594or to trigger a PIM assert message. 595However, it should not be delivered for interfaces that are not in 596the outgoing interface set, and that are not expecting to 597become an incoming interface. 598Hence, if the 599.Dv MRT_MFC_FLAGS_DISABLE_WRONGVIF 600flag is set for some of the 601interfaces, then a data packet that arrives on that interface for 602that MFC entry will NOT trigger a WRONGVIF signal. 603If that flag is not set, then a signal is triggered (the default action). 604.Pp 605The 606.Dv MRT_MFC_FLAGS_BORDER_VIF 607flag is used to specify whether the Border-bit in PIM 608Register messages should be set (in case when the Register encapsulation 609is performed inside the kernel). 610If it is set for the special PIM Register kernel virtual interface 611(see 612.Xr pim 4 ) , 613the Border-bit in the Register messages sent to the RP will be set. 614.Pp 615The remaining six bits are reserved for future usage. 616.Pp 617The 618.Va mfcc_rp 619field is used to specify the RP address (in case of PIM-SM multicast routing) 620for a multicast 621group G if we want to perform kernel-level PIM Register encapsulation. 622The 623.Va mfcc_rp 624field is used only if the 625.Dv MRT_MFC_RP 626advanced API flag/capability has been successfully set by 627.Fn setsockopt MRT_API_CONFIG . 628.Pp 629.\" 630.\" 3. Kernel-level PIM Register encapsulation 631.\" 632If the 633.Dv MRT_MFC_RP 634flag was successfully set by 635.Fn setsockopt MRT_API_CONFIG , 636then the kernel will attempt to perform 637the PIM Register encapsulation itself instead of sending the 638multicast data packets to user level (inside 639.Dv IGMPMSG_WHOLEPKT 640upcalls) for user-level encapsulation. 641The RP address would be taken from the 642.Va mfcc_rp 643field 644inside the new 645.Vt "struct mfcctl2" . 646However, even if the 647.Dv MRT_MFC_RP 648flag was successfully set, if the 649.Va mfcc_rp 650field was set to 651.Dv INADDR_ANY , 652then the 653kernel will still deliver an 654.Dv IGMPMSG_WHOLEPKT 655upcall with the 656multicast data packet to the user-level process. 657.Pp 658In addition, if the multicast data packet is too large to fit within 659a single IP packet after the PIM Register encapsulation (e.g., if 660its size was on the order of 65500 bytes), the data packet will be 661fragmented, and then each of the fragments will be encapsulated 662separately. 663Note that typically a multicast data packet can be that 664large only if it was originated locally from the same hosts that 665performs the encapsulation; otherwise the transmission of the 666multicast data packet over Ethernet for example would have 667fragmented it into much smaller pieces. 668.\" 669.\" Note that if this code is ported to IPv6, we may need the kernel to 670.\" perform MTU discovery to the RP, and keep those discoveries inside 671.\" the kernel so the encapsulating router may send back ICMP 672.\" Fragmentation Required if the size of the multicast data packet is 673.\" too large (see "Encapsulating data packets in the Register Tunnel" 674.\" in Section 4.4.1 in the PIM-SM spec 675.\" draft-ietf-pim-sm-v2-new-05.{txt,ps}). 676.\" For IPv4 we may be able to get away without it, but for IPv6 we need 677.\" that. 678.\" 679.\" 4. Mechanism for "multicast bandwidth monitoring and upcalls". 680.\" 681.Pp 682Typically, a multicast routing user-level process would need to know the 683forwarding bandwidth for some data flow. 684For example, the multicast routing process may want to timeout idle MFC 685entries, or in case of PIM-SM it can initiate (S,G) shortest-path switch if 686the bandwidth rate is above a threshold for example. 687.Pp 688The original solution for measuring the bandwidth of a dataflow was 689that a user-level process would periodically 690query the kernel about the number of forwarded packets/bytes per 691(S,G), and then based on those numbers it would estimate whether a source 692has been idle, or whether the source's transmission bandwidth is above a 693threshold. 694That solution is far from being scalable, hence the need for a new 695mechanism for bandwidth monitoring. 696.Pp 697Below is a description of the bandwidth monitoring mechanism. 698.Bl -bullet 699.It 700If the bandwidth of a data flow satisfies some pre-defined filter, 701the kernel delivers an upcall on the multicast routing socket 702to the multicast routing process that has installed that filter. 703.It 704The bandwidth-upcall filters are installed per (S,G). 705There can be 706more than one filter per (S,G). 707.It 708Instead of supporting all possible comparison operations 709(i.e., < <= == != > >= ), there is support only for the 710<= and >= operations, 711because this makes the kernel-level implementation simpler, 712and because practically we need only those two. 713Further, the missing operations can be simulated by secondary 714user-level filtering of those <= and >= filters. 715For example, to simulate !=, then we need to install filter 716.Dq bw <= 0xffffffff , 717and after an 718upcall is received, we need to check whether 719.Dq measured_bw != expected_bw . 720.It 721The bandwidth-upcall mechanism is enabled by 722.Fn setsockopt MRT_API_CONFIG 723for the 724.Dv MRT_MFC_BW_UPCALL 725flag. 726.It 727The bandwidth-upcall filters are added/deleted by the new 728.Fn setsockopt MRT_ADD_BW_UPCALL 729and 730.Fn setsockopt MRT_DEL_BW_UPCALL 731respectively (with the appropriate 732.Vt "struct bw_upcall" 733argument of course). 734.El 735.Pp 736From application point of view, a developer needs to know about 737the following: 738.Bd -literal 739/* 740 * Structure for installing or delivering an upcall if the 741 * measured bandwidth is above or below a threshold. 742 * 743 * User programs (e.g. daemons) may have a need to know when the 744 * bandwidth used by some data flow is above or below some threshold. 745 * This interface allows the userland to specify the threshold (in 746 * bytes and/or packets) and the measurement interval. Flows are 747 * all packet with the same source and destination IP address. 748 * At the moment the code is only used for multicast destinations 749 * but there is nothing that prevents its use for unicast. 750 * 751 * The measurement interval cannot be shorter than some Tmin (currently, 3s). 752 * The threshold is set in packets and/or bytes per_interval. 753 * 754 * Measurement works as follows: 755 * 756 * For >= measurements: 757 * The first packet marks the start of a measurement interval. 758 * During an interval we count packets and bytes, and when we 759 * pass the threshold we deliver an upcall and we are done. 760 * The first packet after the end of the interval resets the 761 * count and restarts the measurement. 762 * 763 * For <= measurement: 764 * We start a timer to fire at the end of the interval, and 765 * then for each incoming packet we count packets and bytes. 766 * When the timer fires, we compare the value with the threshold, 767 * schedule an upcall if we are below, and restart the measurement 768 * (reschedule timer and zero counters). 769 */ 770 771struct bw_data { 772 struct timeval b_time; 773 uint64_t b_packets; 774 uint64_t b_bytes; 775}; 776 777struct bw_upcall { 778 struct in_addr bu_src; /* source address */ 779 struct in_addr bu_dst; /* destination address */ 780 uint32_t bu_flags; /* misc flags (see below) */ 781#define BW_UPCALL_UNIT_PACKETS (1 << 0) /* threshold (in packets) */ 782#define BW_UPCALL_UNIT_BYTES (1 << 1) /* threshold (in bytes) */ 783#define BW_UPCALL_GEQ (1 << 2) /* upcall if bw >= threshold */ 784#define BW_UPCALL_LEQ (1 << 3) /* upcall if bw <= threshold */ 785#define BW_UPCALL_DELETE_ALL (1 << 4) /* delete all upcalls for s,d*/ 786 struct bw_data bu_threshold; /* the bw threshold */ 787 struct bw_data bu_measured; /* the measured bw */ 788}; 789 790/* max. number of upcalls to deliver together */ 791#define BW_UPCALLS_MAX 128 792/* min. threshold time interval for bandwidth measurement */ 793#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_SEC 3 794#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_USEC 0 795.Ed 796.Pp 797The 798.Vt bw_upcall 799structure is used as an argument to 800.Fn setsockopt MRT_ADD_BW_UPCALL 801and 802.Fn setsockopt MRT_DEL_BW_UPCALL . 803Each 804.Fn setsockopt MRT_ADD_BW_UPCALL 805installs a filter in the kernel 806for the source and destination address in the 807.Vt bw_upcall 808argument, 809and that filter will trigger an upcall according to the following 810pseudo-algorithm: 811.Bd -literal 812 if (bw_upcall_oper IS ">=") { 813 if (((bw_upcall_unit & PACKETS == PACKETS) && 814 (measured_packets >= threshold_packets)) || 815 ((bw_upcall_unit & BYTES == BYTES) && 816 (measured_bytes >= threshold_bytes))) 817 SEND_UPCALL("measured bandwidth is >= threshold"); 818 } 819 if (bw_upcall_oper IS "<=" && measured_interval >= threshold_interval) { 820 if (((bw_upcall_unit & PACKETS == PACKETS) && 821 (measured_packets <= threshold_packets)) || 822 ((bw_upcall_unit & BYTES == BYTES) && 823 (measured_bytes <= threshold_bytes))) 824 SEND_UPCALL("measured bandwidth is <= threshold"); 825 } 826.Ed 827.Pp 828In the same 829.Vt bw_upcall 830the unit can be specified in both BYTES and PACKETS. 831However, the GEQ and LEQ flags are mutually exclusive. 832.Pp 833Basically, an upcall is delivered if the measured bandwidth is >= or 834<= the threshold bandwidth (within the specified measurement 835interval). 836For practical reasons, the smallest value for the measurement 837interval is 3 seconds. 838If smaller values are allowed, then the bandwidth 839estimation may be less accurate, or the potentially very high frequency 840of the generated upcalls may introduce too much overhead. 841For the >= operation, the answer may be known before the end of 842.Va threshold_interval , 843therefore the upcall may be delivered earlier. 844For the <= operation however, we must wait 845until the threshold interval has expired to know the answer. 846.Pp 847Example of usage: 848.Bd -literal 849struct bw_upcall bw_upcall; 850/* Assign all bw_upcall fields as appropriate */ 851memset(&bw_upcall, 0, sizeof(bw_upcall)); 852memcpy(&bw_upcall.bu_src, &source, sizeof(bw_upcall.bu_src)); 853memcpy(&bw_upcall.bu_dst, &group, sizeof(bw_upcall.bu_dst)); 854bw_upcall.bu_threshold.b_data = threshold_interval; 855bw_upcall.bu_threshold.b_packets = threshold_packets; 856bw_upcall.bu_threshold.b_bytes = threshold_bytes; 857if (is_threshold_in_packets) 858 bw_upcall.bu_flags |= BW_UPCALL_UNIT_PACKETS; 859if (is_threshold_in_bytes) 860 bw_upcall.bu_flags |= BW_UPCALL_UNIT_BYTES; 861do { 862 if (is_geq_upcall) { 863 bw_upcall.bu_flags |= BW_UPCALL_GEQ; 864 break; 865 } 866 if (is_leq_upcall) { 867 bw_upcall.bu_flags |= BW_UPCALL_LEQ; 868 break; 869 } 870 return (ERROR); 871} while (0); 872setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_BW_UPCALL, 873 (void *)&bw_upcall, sizeof(bw_upcall)); 874.Ed 875.Pp 876To delete a single filter, then use 877.Dv MRT_DEL_BW_UPCALL , 878and the fields of bw_upcall must be set 879exactly same as when 880.Dv MRT_ADD_BW_UPCALL 881was called. 882.Pp 883To delete all bandwidth filters for a given (S,G), then 884only the 885.Va bu_src 886and 887.Va bu_dst 888fields in 889.Vt "struct bw_upcall" 890need to be set, and then just set only the 891.Dv BW_UPCALL_DELETE_ALL 892flag inside field 893.Va bw_upcall.bu_flags . 894.Pp 895The bandwidth upcalls are received by aggregating them in the new upcall 896message: 897.Bd -literal 898#define IGMPMSG_BW_UPCALL 4 /* BW monitoring upcall */ 899.Ed 900.Pp 901This message is an array of 902.Vt "struct bw_upcall" 903elements (up to 904.Dv BW_UPCALLS_MAX 905= 128). 906The upcalls are 907delivered when there are 128 pending upcalls, or when 1 second has 908expired since the previous upcall (whichever comes first). 909In an 910.Vt "struct upcall" 911element, the 912.Va bu_measured 913field is filled-in to 914indicate the particular measured values. 915However, because of the way 916the particular intervals are measured, the user should be careful how 917.Va bu_measured.b_time 918is used. 919For example, if the 920filter is installed to trigger an upcall if the number of packets 921is >= 1, then 922.Va bu_measured 923may have a value of zero in the upcalls after the 924first one, because the measured interval for >= filters is 925.Dq clocked 926by the forwarded packets. 927Hence, this upcall mechanism should not be used for measuring 928the exact value of the bandwidth of the forwarded data. 929To measure the exact bandwidth, the user would need to 930get the forwarded packets statistics with the 931.Fn ioctl SIOCGETSGCNT 932mechanism 933(see the 934.Sx Programming Guide 935section) . 936.Pp 937Note that the upcalls for a filter are delivered until the specific 938filter is deleted, but no more frequently than once per 939.Va bu_threshold.b_time . 940For example, if the filter is specified to 941deliver a signal if bw >= 1 packet, the first packet will trigger a 942signal, but the next upcall will be triggered no earlier than 943.Va bu_threshold.b_time 944after the previous upcall. 945.\" 946.Sh SEE ALSO 947.Xr altq 4 , 948.Xr dummynet 4 , 949.Xr getsockopt 2 , 950.Xr gif 4 , 951.Xr gre 4 , 952.Xr recvfrom 2 , 953.Xr recvmsg 2 , 954.Xr setsockopt 2 , 955.Xr socket 2 , 956.Xr icmp6 4 , 957.Xr inet 4 , 958.Xr inet6 4 , 959.Xr intro 4 , 960.Xr ip 4 , 961.Xr ip6 4 , 962.Xr pim 4 963.\" 964.Sh AUTHORS 965.An -nosplit 966The original multicast code was written by 967.An David Waitzman 968(BBN Labs), 969and later modified by the following individuals: 970.An Steve Deering 971(Stanford), 972.An Mark J. Steiglitz 973(Stanford), 974.An Van Jacobson 975(LBL), 976.An Ajit Thyagarajan 977(PARC), 978.An Bill Fenner 979(PARC). 980The IPv6 multicast support was implemented by the KAME project 981.Pq Pa http://www.kame.net , 982and was based on the IPv4 multicast code. 983The advanced multicast API and the multicast bandwidth 984monitoring were implemented by 985.An Pavlin Radoslavov 986(ICSI) 987in collaboration with 988.An Chris Brown 989(NextHop). 990.Pp 991This manual page was written by 992.An Pavlin Radoslavov 993(ICSI). 994