1.\" Copyright (c) 2001-2003 International Computer Science Institute 2.\" 3.\" Permission is hereby granted, free of charge, to any person obtaining a 4.\" copy of this software and associated documentation files (the "Software"), 5.\" to deal in the Software without restriction, including without limitation 6.\" the rights to use, copy, modify, merge, publish, distribute, sublicense, 7.\" and/or sell copies of the Software, and to permit persons to whom the 8.\" Software is furnished to do so, subject to the following conditions: 9.\" 10.\" The above copyright notice and this permission notice shall be included in 11.\" all copies or substantial portions of the Software. 12.\" 13.\" The names and trademarks of copyright holders may not be used in 14.\" advertising or publicity pertaining to the software without specific 15.\" prior permission. Title to copyright in this software and any associated 16.\" documentation will at all times remain with the copyright holders. 17.\" 18.\" THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19.\" IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20.\" FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21.\" AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22.\" LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 23.\" FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 24.\" DEALINGS IN THE SOFTWARE. 25.\" 26.\" $FreeBSD: /repoman/r/ncvs/src/share/man/man4/multicast.4,v 1.1 2003/10/17 15:12:01 bmah Exp $ 27.\" $DragonFly: src/share/man/man4/multicast.4,v 1.6 2007/11/24 18:47:07 swildner Exp $ 28.\" 29.Dd September 4, 2003 30.Dt MULTICAST 4 31.Os 32.\" 33.Sh NAME 34.Nm multicast 35.Nd Multicast Routing 36.\" 37.Sh SYNOPSIS 38.Cd "options MROUTING" 39.Pp 40.In sys/types.h 41.In sys/socket.h 42.In netinet/in.h 43.In net/ip_mroute/ip_mroute.h 44.In netinet6/ip6_mroute.h 45.Ft int 46.Fn getsockopt "int s" IPPROTO_IP MRT_INIT "void *optval" "socklen_t *optlen" 47.Ft int 48.Fn setsockopt "int s" IPPROTO_IP MRT_INIT "const void *optval" "socklen_t optlen" 49.Ft int 50.Fn getsockopt "int s" IPPROTO_IPV6 MRT6_INIT "void *optval" "socklen_t *optlen" 51.Ft int 52.Fn setsockopt "int s" IPPROTO_IPV6 MRT6_INIT "const void *optval" "socklen_t optlen" 53.Sh DESCRIPTION 54.Tn "Multicast routing" 55is used to efficiently propagate data 56packets to a set of multicast listeners in multipoint networks. 57If unicast is used to replicate the data to all listeners, 58then some of the network links may carry multiple copies of the same 59data packets. 60With multicast routing, the overhead is reduced to one copy 61(at most) per network link. 62.Pp 63All multicast-capable routers must run a common multicast routing 64protocol. 65The Distance Vector Multicast Routing Protocol (DVMRP) 66was the first developed multicast routing protocol. 67Later, other protocols such as Multicast Extensions to OSPF (MOSPF), 68Core Based Trees (CBT), 69Protocol Independent Multicast - Sparse Mode (PIM-SM), 70and Protocol Independent Multicast - Dense Mode (PIM-DM) 71were developed as well. 72.Pp 73To start multicast routing, 74the user must enable multicast forwarding in the kernel 75(see 76.Sx SYNOPSIS 77about the kernel configuration options), 78and must run a multicast routing capable user-level process. 79From developer's point of view, 80the programming guide described in the 81.Sx "Programming Guide" 82section should be used to control the multicast forwarding in the kernel. 83.\" 84.Ss Programming Guide 85This section provides information about the basic multicast routing API. 86The so-called 87.Dq advanced multicast API 88is described in the 89.Sx "Advanced Multicast API Programming Guide" 90section. 91.Pp 92First, a multicast routing socket must be open. 93That socket would be used 94to control the multicast forwarding in the kernel. 95Note that most operations below require certain privilege 96(i.e., root privilege): 97.Pp 98.Bd -literal 99/* IPv4 */ 100int mrouter_s4; 101mrouter_s4 = socket(AF_INET, SOCK_RAW, IPPROTO_IGMP); 102.Ed 103.Pp 104.Bd -literal 105int mrouter_s6; 106mrouter_s6 = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); 107.Ed 108.Pp 109Note that if the router needs to open an IGMP or ICMPv6 socket 110(in case of IPv4 and IPv6 respectively) 111for sending or receiving of IGMP or MLD multicast group membership messages, 112then the same mrouter_s4 or mrouter_s6 sockets should be used 113for sending and receiving respectively IGMP or MLD messages. 114In case of BSD-derived kernel, it may be possible to open separate sockets 115for IGMP or MLD messages only. 116However, some other kernels (e.g., Linux) require that the multicast 117routing socket must be used for sending and receiving of IGMP or MLD 118messages. 119Therefore, for portability reason the multicast 120routing socket should be reused for IGMP and MLD messages as well. 121.Pp 122After the multicast routing socket is open, it can be used to enable 123or disable multicast forwarding in the kernel: 124.Bd -literal 125/* IPv4 */ 126int v = 1; /* 1 to enable, or 0 to disable */ 127setsockopt(mrouter_s4, IPPROTO_IP, MRT_INIT, (void *)&v, sizeof(v)); 128.Ed 129.Pp 130.Bd -literal 131/* IPv6 */ 132int v = 1; /* 1 to enable, or 0 to disable */ 133setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_INIT, (void *)&v, sizeof(v)); 134\&... 135/* If necessary, filter all ICMPv6 messages */ 136struct icmp6_filter filter; 137ICMP6_FILTER_SETBLOCKALL(&filter); 138setsockopt(mrouter_s6, IPPROTO_ICMPV6, ICMP6_FILTER, (void *)&filter, 139 sizeof(filter)); 140.Ed 141.Pp 142After multicast forwarding is enabled, the multicast routing socket 143can be used to enable PIM processing in the kernel if we are running PIM-SM or 144PIM-DM 145(see 146.Xr pim 4 ) . 147.Pp 148For each network interface (e.g., physical or a virtual tunnel) 149that would be used for multicast forwarding, a corresponding 150multicast interface must be added to the kernel: 151.Bd -literal 152/* IPv4 */ 153struct vifctl vc; 154memset(&vc, 0, sizeof(vc)); 155/* Assign all vifctl fields as appropriate */ 156vc.vifc_vifi = vif_index; 157vc.vifc_flags = vif_flags; 158vc.vifc_threshold = min_ttl_threshold; 159vc.vifc_rate_limit = max_rate_limit; 160memcpy(&vc.vifc_lcl_addr, &vif_local_address, sizeof(vc.vifc_lcl_addr)); 161if (vc.vifc_flags & VIFF_TUNNEL) 162 memcpy(&vc.vifc_rmt_addr, &vif_remote_address, 163 sizeof(vc.vifc_rmt_addr)); 164setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_VIF, (void *)&vc, 165 sizeof(vc)); 166.Ed 167.Pp 168The 169.Dq vif_index 170must be unique per vif. 171The 172.Dq vif_flags 173contains the 174.Dq VIFF_* 175flags as defined in 176.In net/ip_mroute/ip_mroute.h . 177The 178.Dq min_ttl_threshold 179contains the minimum TTL a multicast data packet must have to be 180forwarded on that vif. 181Typically, it would have value of 1. 182The 183.Dq max_rate_limit 184contains the maximum rate (in bits/s) of the multicast data packets forwarded 185on that vif. 186Value of 0 means no limit. 187The 188.Dq vif_local_address 189contains the local IP address of the corresponding local interface. 190The 191.Dq vif_remote_address 192contains the remote IP address in case of DVMRP multicast tunnels. 193.Pp 194.Bd -literal 195/* IPv6 */ 196struct mif6ctl mc; 197memset(&mc, 0, sizeof(mc)); 198/* Assign all mif6ctl fields as appropriate */ 199mc.mif6c_mifi = mif_index; 200mc.mif6c_flags = mif_flags; 201mc.mif6c_pifi = pif_index; 202setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_ADD_MIF, (void *)&mc, 203 sizeof(mc)); 204.Ed 205.Pp 206The 207.Dq mif_index 208must be unique per vif. 209The 210.Dq mif_flags 211contains the 212.Dq MIFF_* 213flags as defined in 214.In netinet6/ip6_mroute.h . 215The 216.Dq pif_index 217is the physical interface index of the corresponding local interface. 218.Pp 219A multicast interface is deleted by: 220.Bd -literal 221/* IPv4 */ 222vifi_t vifi = vif_index; 223setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_VIF, (void *)&vifi, 224 sizeof(vifi)); 225.Ed 226.Pp 227.Bd -literal 228/* IPv6 */ 229mifi_t mifi = mif_index; 230setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_DEL_MIF, (void *)&mifi, 231 sizeof(mifi)); 232.Ed 233.Pp 234After the multicast forwarding is enabled, and the multicast virtual 235interfaces are 236added, the kernel may deliver upcall messages (also called signals 237later in this text) on the multicast routing socket that was open 238earlier with 239.Dq MRT_INIT 240or 241.Dq MRT6_INIT . 242The IPv4 upcalls have 243.Dq struct igmpmsg 244header (see 245.In net/ip_mroute/ip_mroute.h ) 246with field 247.Dq im_mbz 248set to zero. 249Note that this header follows the structure of 250.Dq struct ip 251with the protocol field 252.Dq ip_p 253set to zero. 254The IPv6 upcalls have 255.Dq struct mrt6msg 256header (see 257.In netinet6/ip6_mroute.h ) 258with field 259.Dq im6_mbz 260set to zero. 261Note that this header follows the structure of 262.Dq struct ip6_hdr 263with the next header field 264.Dq ip6_nxt 265set to zero. 266.Pp 267The upcall header contains field 268.Dq im_msgtype 269and 270.Dq im6_msgtype 271with the type of the upcall 272.Dq IGMPMSG_* 273and 274.Dq MRT6MSG_* 275for IPv4 and IPv6 respectively. 276The values of the rest of the upcall header fields 277and the body of the upcall message depend on the particular upcall type. 278.Pp 279If the upcall message type is 280.Dq IGMPMSG_NOCACHE 281or 282.Dq MRT6MSG_NOCACHE , 283this is an indication that a multicast packet has reached the multicast 284router, but the router has no forwarding state for that packet. 285Typically, the upcall would be a signal for the multicast routing 286user-level process to install the appropriate Multicast Forwarding 287Cache (MFC) entry in the kernel. 288.Pp 289A MFC entry is added by: 290.Bd -literal 291/* IPv4 */ 292struct mfcctl mc; 293memset(&mc, 0, sizeof(mc)); 294memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 295memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 296mc.mfcc_parent = iif_index; 297for (i = 0; i < maxvifs; i++) 298 mc.mfcc_ttls[i] = oifs_ttl[i]; 299setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_MFC, 300 (void *)&mc, sizeof(mc)); 301.Ed 302.Pp 303.Bd -literal 304/* IPv6 */ 305struct mf6cctl mc; 306memset(&mc, 0, sizeof(mc)); 307memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 308memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 309mc.mf6cc_parent = iif_index; 310for (i = 0; i < maxvifs; i++) 311 if (oifs_ttl[i] > 0) 312 IF_SET(i, &mc.mf6cc_ifset); 313setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_ADD_MFC, 314 (void *)&mc, sizeof(mc)); 315.Ed 316.Pp 317The 318.Dq source_addr 319and 320.Dq group_addr 321are the source and group address of the multicast packet (as set 322in the upcall message). 323The 324.Dq iif_index 325is the virtual interface index of the multicast interface the multicast 326packets for this specific source and group address should be received on. 327The 328.Dq oifs_ttl[] 329array contains the minimum TTL (per interface) a multicast packet 330should have to be forwarded on an outgoing interface. 331If the TTL value is zero, the corresponding interface is not included 332in the set of outgoing interfaces. 333Note that in case of IPv6 only the set of outgoing interfaces can 334be specified. 335.Pp 336A MFC entry is deleted by: 337.Bd -literal 338/* IPv4 */ 339struct mfcctl mc; 340memset(&mc, 0, sizeof(mc)); 341memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 342memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 343setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_MFC, 344 (void *)&mc, sizeof(mc)); 345.Ed 346.Pp 347.Bd -literal 348/* IPv6 */ 349struct mf6cctl mc; 350memset(&mc, 0, sizeof(mc)); 351memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 352memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 353setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_DEL_MFC, 354 (void *)&mc, sizeof(mc)); 355.Ed 356.Pp 357The following method can be used to get various statistics per 358installed MFC entry in the kernel (e.g., the number of forwarded 359packets per source and group address): 360.Bd -literal 361/* IPv4 */ 362struct sioc_sg_req sgreq; 363memset(&sgreq, 0, sizeof(sgreq)); 364memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 365memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 366ioctl(mrouter_s4, SIOCGETSGCNT, &sgreq); 367.Ed 368.Pp 369.Bd -literal 370/* IPv6 */ 371struct sioc_sg_req6 sgreq; 372memset(&sgreq, 0, sizeof(sgreq)); 373memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 374memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 375ioctl(mrouter_s6, SIOCGETSGCNT_IN6, &sgreq); 376.Ed 377.Pp 378The following method can be used to get various statistics per 379multicast virtual interface in the kernel (e.g., the number of forwarded 380packets per interface): 381.Bd -literal 382/* IPv4 */ 383struct sioc_vif_req vreq; 384memset(&vreq, 0, sizeof(vreq)); 385vreq.vifi = vif_index; 386ioctl(mrouter_s4, SIOCGETVIFCNT, &vreq); 387.Ed 388.Pp 389.Bd -literal 390/* IPv6 */ 391struct sioc_mif_req6 mreq; 392memset(&mreq, 0, sizeof(mreq)); 393mreq.mifi = vif_index; 394ioctl(mrouter_s6, SIOCGETMIFCNT_IN6, &mreq); 395.Ed 396.Ss Advanced Multicast API Programming Guide 397If we want to add new features in the kernel, it becomes difficult 398to preserve backward compatibility (binary and API), 399and at the same time to allow user-level processes to take advantage of 400the new features (if the kernel supports them). 401.Pp 402One of the mechanisms that allows us to preserve the backward 403compatibility is a sort of negotiation 404between the user-level process and the kernel: 405.Bl -enum 406.It 407The user-level process tries to enable in the kernel the set of new 408features (and the corresponding API) it would like to use. 409.It 410The kernel returns the (sub)set of features it knows about 411and is willing to be enabled. 412.It 413The user-level process uses only that set of features 414the kernel has agreed on. 415.El 416.\" 417.Pp 418To support backward compatibility, if the user-level process doesn't 419ask for any new features, the kernel defaults to the basic 420multicast API (see the 421.Sx "Programming Guide" 422section). 423.\" XXX: edit as appropriate after the advanced multicast API is 424.\" supported under IPv6 425Currently, the advanced multicast API exists only for IPv4; 426in the future there will be IPv6 support as well. 427.Pp 428Below is a summary of the expandable API solution. 429Note that all new options and structures are defined 430in 431.In net/ip_mroute/ip_mroute.h 432and 433.In netinet6/ip6_mroute.h , 434unless stated otherwise. 435.Pp 436The user-level process uses new get/setsockopt() options to 437perform the API features negotiation with the kernel. 438This negotiation must be performed right after the multicast routing 439socket is open. 440The set of desired/allowed features is stored in a bitset 441(currently, in uint32_t; i.e., maximum of 32 new features). 442The new get/setsockopt() options are 443.Dq MRT_API_SUPPORT 444and 445.Dq MRT_API_CONFIG . 446Example: 447.Bd -literal 448uint32_t v; 449getsockopt(sock, IPPROTO_IP, MRT_API_SUPPORT, (void *)&v, sizeof(v)); 450.Ed 451.Pp 452would set in 453.Dq v 454the pre-defined bits that the kernel API supports. 455The eight least significant bits in uint32_t are same as the 456eight possible flags 457.Dq MRT_MFC_FLAGS_* 458that can be used in 459.Dq mfcc_flags 460as part of the new definition of 461.Dq struct mfcctl 462(see below about those flags), which leaves 24 flags for other new features. 463The value returned by getsockopt(MRT_API_SUPPORT) is read-only; in other 464words, setsockopt(MRT_API_SUPPORT) would fail. 465.Pp 466To modify the API, and to set some specific feature in the kernel, then: 467.Bd -literal 468uint32_t v = MRT_MFC_FLAGS_DISABLE_WRONGVIF; 469if (setsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)) 470 != 0) { 471 return (ERROR); 472} 473if (v & MRT_MFC_FLAGS_DISABLE_WRONGVIF) 474 return (OK); /* Success */ 475else 476 return (ERROR); 477.Ed 478.Pp 479In other words, when setsockopt(MRT_API_CONFIG) is called, the 480argument to it specifies the desired set of features to 481be enabled in the API and the kernel. 482The return value in 483.Dq v 484is the actual (sub)set of features that were enabled in the kernel. 485To obtain later the same set of features that were enabled, then: 486.Bd -literal 487getsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)); 488.Ed 489.Pp 490The set of enabled features is global. 491In other words, setsockopt(MRT_API_CONFIG) 492should be called right after setsockopt(MRT_INIT). 493.Pp 494Currently, the following set of new features is defined: 495.Bd -literal 496#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 497#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 498#define MRT_MFC_RP (1 << 8) /* enable RP address */ 499#define MRT_MFC_BW_UPCALL (1 << 9) /* enable bw upcalls */ 500.Ed 501.\" .Pp 502.\" In the future there might be: 503.\" .Bd -literal 504.\" #define MRT_MFC_GROUP_SPECIFIC (1 << 10) /* allow (*,G) MFC entries */ 505.\" .Ed 506.\" .Pp 507.\" to allow (*,G) MFC entries (i.e., group-specific entries) in the kernel. 508.\" For now this is left-out until it is clear whether 509.\" (*,G) MFC support is the preferred solution instead of something more generic 510.\" solution for example. 511.\" 512.\" 2. The newly defined struct mfcctl2. 513.\" 514.Pp 515The advanced multicast API uses a newly defined 516.Dq struct mfcctl2 517instead of the traditional 518.Dq struct mfcctl . 519The original 520.Dq struct mfcctl 521is kept as is. 522The new 523.Dq struct mfcctl2 524is: 525.Bd -literal 526/* 527 * The new argument structure for MRT_ADD_MFC and MRT_DEL_MFC overlays 528 * and extends the old struct mfcctl. 529 */ 530struct mfcctl2 { 531 /* the mfcctl fields */ 532 struct in_addr mfcc_origin; /* ip origin of mcasts */ 533 struct in_addr mfcc_mcastgrp; /* multicast group associated*/ 534 vifi_t mfcc_parent; /* incoming vif */ 535 u_char mfcc_ttls[MAXVIFS];/* forwarding ttls on vifs */ 536 537 /* extension fields */ 538 uint8_t mfcc_flags[MAXVIFS];/* the MRT_MFC_FLAGS_* flags*/ 539 struct in_addr mfcc_rp; /* the RP address */ 540}; 541.Ed 542.Pp 543The new fields are 544.Dq mfcc_flags[MAXVIFS] 545and 546.Dq mfcc_rp . 547Note that for compatibility reasons they are added at the end. 548.Pp 549The 550.Dq mfcc_flags[MAXVIFS] 551field is used to set various flags per 552interface per (S,G) entry. 553Currently, the defined flags are: 554.Bd -literal 555#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 556#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 557.Ed 558.Pp 559The 560.Dq MRT_MFC_FLAGS_DISABLE_WRONGVIF 561flag is used to explicitly disable the 562.Dq IGMPMSG_WRONGVIF 563kernel signal at the (S,G) granularity if a multicast data packet 564arrives on the wrong interface. 565Usually, this signal is used to 566complete the shortest-path switch in case of PIM-SM multicast routing, 567or to trigger a PIM assert message. 568However, it should not be delivered for interfaces that are not in 569the outgoing interface set, and that are not expecting to 570become an incoming interface. 571Hence, if the 572.Dq MRT_MFC_FLAGS_DISABLE_WRONGVIF 573flag is set for some of the 574interfaces, then a data packet that arrives on that interface for 575that MFC entry will NOT trigger a WRONGVIF signal. 576If that flag is not set, then a signal is triggered (the default action). 577.Pp 578The 579.Dq MRT_MFC_FLAGS_BORDER_VIF 580flag is used to specify whether the Border-bit in PIM 581Register messages should be set (in case when the Register encapsulation 582is performed inside the kernel). 583If it is set for the special PIM Register kernel virtual interface 584(see 585.Xr pim 4 ) , 586the Border-bit in the Register messages sent to the RP will be set. 587.Pp 588The remaining six bits are reserved for future usage. 589.Pp 590The 591.Dq mfcc_rp 592field is used to specify the RP address (in case of PIM-SM multicast routing) 593for a multicast 594group G if we want to perform kernel-level PIM Register encapsulation. 595The 596.Dq mfcc_rp 597field is used only if the 598.Dq MRT_MFC_RP 599advanced API flag/capability has been successfully set by 600setsockopt(MRT_API_CONFIG). 601.Pp 602.\" 603.\" 3. Kernel-level PIM Register encapsulation 604.\" 605If the 606.Dq MRT_MFC_RP 607flag was successfully set by 608setsockopt(MRT_API_CONFIG), then the kernel will attempt to perform 609the PIM Register encapsulation itself instead of sending the 610multicast data packets to user level (inside IGMPMSG_WHOLEPKT 611upcalls) for user-level encapsulation. 612The RP address would be taken from the 613.Dq mfcc_rp 614field 615inside the new 616.Dq struct mfcctl2 . 617However, even if the 618.Dq MRT_MFC_RP 619flag was successfully set, if the 620.Dq mfcc_rp 621field was set to 622.Dq INADDR_ANY , 623then the 624kernel will still deliver an IGMPMSG_WHOLEPKT upcall with the 625multicast data packet to the user-level process. 626.Pp 627In addition, if the multicast data packet is too large to fit within 628a single IP packet after the PIM Register encapsulation (e.g., if 629its size was on the order of 65500 bytes), the data packet will be 630fragmented, and then each of the fragments will be encapsulated 631separately. 632Note that typically a multicast data packet can be that 633large only if it was originated locally from the same hosts that 634performs the encapsulation; otherwise the transmission of the 635multicast data packet over Ethernet for example would have 636fragmented it into much smaller pieces. 637.\" 638.\" Note that if this code is ported to IPv6, we may need the kernel to 639.\" perform MTU discovery to the RP, and keep those discoveries inside 640.\" the kernel so the encapsulating router may send back ICMP 641.\" Fragmentation Required if the size of the multicast data packet is 642.\" too large (see "Encapsulating data packets in the Register Tunnel" 643.\" in Section 4.4.1 in the PIM-SM spec 644.\" draft-ietf-pim-sm-v2-new-05.{txt,ps}). 645.\" For IPv4 we may be able to get away without it, but for IPv6 we need 646.\" that. 647.\" 648.\" 4. Mechanism for "multicast bandwidth monitoring and upcalls". 649.\" 650.Pp 651Typically, a multicast routing user-level process would need to know the 652forwarding bandwidth for some data flow. 653For example, the multicast routing process may want to timeout idle MFC 654entries, or in case of PIM-SM it can initiate (S,G) shortest-path switch if 655the bandwidth rate is above a threshold for example. 656.Pp 657The original solution for measuring the bandwidth of a dataflow was 658that a user-level process would periodically 659query the kernel about the number of forwarded packets/bytes per 660(S,G), and then based on those numbers it would estimate whether a source 661has been idle, or whether the source's transmission bandwidth is above a 662threshold. 663That solution is far from being scalable, hence the need for a new 664mechanism for bandwidth monitoring. 665.Pp 666Below is a description of the bandwidth monitoring mechanism. 667.Bl -bullet 668.It 669If the bandwidth of a data flow satisfies some pre-defined filter, 670the kernel delivers an upcall on the multicast routing socket 671to the multicast routing process that has installed that filter. 672.It 673The bandwidth-upcall filters are installed per (S,G). There can be 674more than one filter per (S,G). 675.It 676Instead of supporting all possible comparison operations 677(i.e., < <= == != > >= ), there is support only for the 678<= and >= operations, 679because this makes the kernel-level implementation simpler, 680and because practically we need only those two. 681Further, the missing operations can be simulated by secondary 682user-level filtering of those <= and >= filters. 683For example, to simulate !=, then we need to install filter 684.Dq bw <= 0xffffffff , 685and after an 686upcall is received, we need to check whether 687.Dq measured_bw != expected_bw . 688.It 689The bandwidth-upcall mechanism is enabled by 690setsockopt(MRT_API_CONFIG) for the MRT_MFC_BW_UPCALL flag. 691.It 692The bandwidth-upcall filters are added/deleted by the new 693setsockopt(MRT_ADD_BW_UPCALL) and setsockopt(MRT_DEL_BW_UPCALL) 694respectively (with the appropriate 695.Dq struct bw_upcall 696argument of course). 697.El 698.Pp 699From application point of view, a developer needs to know about 700the following: 701.Bd -literal 702/* 703 * Structure for installing or delivering an upcall if the 704 * measured bandwidth is above or below a threshold. 705 * 706 * User programs (e.g. daemons) may have a need to know when the 707 * bandwidth used by some data flow is above or below some threshold. 708 * This interface allows the userland to specify the threshold (in 709 * bytes and/or packets) and the measurement interval. Flows are 710 * all packet with the same source and destination IP address. 711 * At the moment the code is only used for multicast destinations 712 * but there is nothing that prevents its use for unicast. 713 * 714 * The measurement interval cannot be shorter than some Tmin (currently, 3s). 715 * The threshold is set in packets and/or bytes per_interval. 716 * 717 * Measurement works as follows: 718 * 719 * For >= measurements: 720 * The first packet marks the start of a measurement interval. 721 * During an interval we count packets and bytes, and when we 722 * pass the threshold we deliver an upcall and we are done. 723 * The first packet after the end of the interval resets the 724 * count and restarts the measurement. 725 * 726 * For <= measurement: 727 * We start a timer to fire at the end of the interval, and 728 * then for each incoming packet we count packets and bytes. 729 * When the timer fires, we compare the value with the threshold, 730 * schedule an upcall if we are below, and restart the measurement 731 * (reschedule timer and zero counters). 732 */ 733 734struct bw_data { 735 struct timeval b_time; 736 uint64_t b_packets; 737 uint64_t b_bytes; 738}; 739 740struct bw_upcall { 741 struct in_addr bu_src; /* source address */ 742 struct in_addr bu_dst; /* destination address */ 743 uint32_t bu_flags; /* misc flags (see below) */ 744#define BW_UPCALL_UNIT_PACKETS (1 << 0) /* threshold (in packets) */ 745#define BW_UPCALL_UNIT_BYTES (1 << 1) /* threshold (in bytes) */ 746#define BW_UPCALL_GEQ (1 << 2) /* upcall if bw >= threshold */ 747#define BW_UPCALL_LEQ (1 << 3) /* upcall if bw <= threshold */ 748#define BW_UPCALL_DELETE_ALL (1 << 4) /* delete all upcalls for s,d*/ 749 struct bw_data bu_threshold; /* the bw threshold */ 750 struct bw_data bu_measured; /* the measured bw */ 751}; 752 753/* max. number of upcalls to deliver together */ 754#define BW_UPCALLS_MAX 128 755/* min. threshold time interval for bandwidth measurement */ 756#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_SEC 3 757#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_USEC 0 758.Ed 759.Pp 760The 761.Dq bw_upcall 762structure is used as an argument to 763setsockopt(MRT_ADD_BW_UPCALL) and setsockopt(MRT_DEL_BW_UPCALL). 764Each setsockopt(MRT_ADD_BW_UPCALL) installs a filter in the kernel 765for the source and destination address in the 766.Dq bw_upcall 767argument, 768and that filter will trigger an upcall according to the following 769pseudo-algorithm: 770.Bd -literal 771 if (bw_upcall_oper IS ">=") { 772 if (((bw_upcall_unit & PACKETS == PACKETS) && 773 (measured_packets >= threshold_packets)) || 774 ((bw_upcall_unit & BYTES == BYTES) && 775 (measured_bytes >= threshold_bytes))) 776 SEND_UPCALL("measured bandwidth is >= threshold"); 777 } 778 if (bw_upcall_oper IS "<=" && measured_interval >= threshold_interval) { 779 if (((bw_upcall_unit & PACKETS == PACKETS) && 780 (measured_packets <= threshold_packets)) || 781 ((bw_upcall_unit & BYTES == BYTES) && 782 (measured_bytes <= threshold_bytes))) 783 SEND_UPCALL("measured bandwidth is <= threshold"); 784 } 785.Ed 786.Pp 787In the same 788.Dq bw_upcall 789the unit can be specified in both BYTES and PACKETS. 790However, the GEQ and LEQ flags are mutually exclusive. 791.Pp 792Basically, an upcall is delivered if the measured bandwidth is >= or 793<= the threshold bandwidth (within the specified measurement 794interval). 795For practical reasons, the smallest value for the measurement 796interval is 3 seconds. 797If smaller values are allowed, then the bandwidth 798estimation may be less accurate, or the potentially very high frequency 799of the generated upcalls may introduce too much overhead. 800For the >= operation, the answer may be known before the end of 801.Dq threshold_interval , 802therefore the upcall may be delivered earlier. 803For the <= operation however, we must wait 804until the threshold interval has expired to know the answer. 805.Pp 806Example of usage: 807.Bd -literal 808struct bw_upcall bw_upcall; 809/* Assign all bw_upcall fields as appropriate */ 810memset(&bw_upcall, 0, sizeof(bw_upcall)); 811memcpy(&bw_upcall.bu_src, &source, sizeof(bw_upcall.bu_src)); 812memcpy(&bw_upcall.bu_dst, &group, sizeof(bw_upcall.bu_dst)); 813bw_upcall.bu_threshold.b_data = threshold_interval; 814bw_upcall.bu_threshold.b_packets = threshold_packets; 815bw_upcall.bu_threshold.b_bytes = threshold_bytes; 816if (is_threshold_in_packets) 817 bw_upcall.bu_flags |= BW_UPCALL_UNIT_PACKETS; 818if (is_threshold_in_bytes) 819 bw_upcall.bu_flags |= BW_UPCALL_UNIT_BYTES; 820do { 821 if (is_geq_upcall) { 822 bw_upcall.bu_flags |= BW_UPCALL_GEQ; 823 break; 824 } 825 if (is_leq_upcall) { 826 bw_upcall.bu_flags |= BW_UPCALL_LEQ; 827 break; 828 } 829 return (ERROR); 830} while (0); 831setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_BW_UPCALL, 832 (void *)&bw_upcall, sizeof(bw_upcall)); 833.Ed 834.Pp 835To delete a single filter, then use MRT_DEL_BW_UPCALL, 836and the fields of bw_upcall must be set 837exactly same as when MRT_ADD_BW_UPCALL was called. 838.Pp 839To delete all bandwidth filters for a given (S,G), then 840only the 841.Dq bu_src 842and 843.Dq bu_dst 844fields in 845.Dq struct bw_upcall 846need to be set, and then just set only the 847.Dq BW_UPCALL_DELETE_ALL 848flag inside field 849.Dq bw_upcall.bu_flags . 850.Pp 851The bandwidth upcalls are received by aggregating them in the new upcall 852message: 853.Bd -literal 854#define IGMPMSG_BW_UPCALL 4 /* BW monitoring upcall */ 855.Ed 856.Pp 857This message is an array of 858.Dq struct bw_upcall 859elements (up to BW_UPCALLS_MAX = 128). 860The upcalls are 861delivered when there are 128 pending upcalls, or when 1 second has 862expired since the previous upcall (whichever comes first). 863In an 864.Dq struct upcall 865element, the 866.Dq bu_measured 867field is filled-in to 868indicate the particular measured values. 869However, because of the way 870the particular intervals are measured, the user should be careful how 871bu_measured.b_time is used. 872For example, if the 873filter is installed to trigger an upcall if the number of packets 874is >= 1, then 875.Dq bu_measured 876may have a value of zero in the upcalls after the 877first one, because the measured interval for >= filters is 878.Dq clocked 879by the forwarded packets. 880Hence, this upcall mechanism should not be used for measuring 881the exact value of the bandwidth of the forwarded data. 882To measure the exact bandwidth, the user would need to 883get the forwarded packets statistics with the ioctl(SIOCGETSGCNT) 884mechanism 885(see the 886.Sx Programming Guide 887section) . 888.Pp 889Note that the upcalls for a filter are delivered until the specific 890filter is deleted, but no more frequently than once per 891.Dq bu_threshold.b_time . 892For example, if the filter is specified to 893deliver a signal if bw >= 1 packet, the first packet will trigger a 894signal, but the next upcall will be triggered no earlier than 895.Dq bu_threshold.b_time 896after the previous upcall. 897.\" 898.Sh SEE ALSO 899.Xr getsockopt 2 , 900.Xr recvfrom 2 , 901.Xr recvmsg 2 , 902.Xr setsockopt 2 , 903.Xr socket 2 , 904.Xr icmp6 4 , 905.Xr inet 4 , 906.Xr inet6 4 , 907.Xr intro 4 , 908.Xr ip 4 , 909.Xr ip6 4 , 910.Xr pim 4 911.\" 912.Sh AUTHORS 913.An -nosplit 914The original multicast code was written by 915.An David Waitzman 916(BBN Labs), and later modified by the following individuals: 917.An Steve Deering 918(Stanford), 919.An Mark J. Steiglitz 920(Stanford), 921.An Van Jacobson 922(LBL), 923.An Ajit Thyagarajan 924(PARC), 925.An Bill Fenner 926(PARC). 927The IPv6 multicast support was implemented by the KAME project 928.Pa ( http://www.kame.net ) , 929and was based on the IPv4 multicast code. 930The advanced multicast API and the multicast bandwidth 931monitoring were implemented by 932.An Pavlin Radoslavov 933(ICSI) in collaboration with 934.An Chris Brown 935(NextHop). 936.Pp 937This manual page was written by 938.An Pavlin Radoslavov 939(ICSI). 940