1.\" Copyright (c) 2001-2003 International Computer Science Institute 2.\" 3.\" Permission is hereby granted, free of charge, to any person obtaining a 4.\" copy of this software and associated documentation files (the "Software"), 5.\" to deal in the Software without restriction, including without limitation 6.\" the rights to use, copy, modify, merge, publish, distribute, sublicense, 7.\" and/or sell copies of the Software, and to permit persons to whom the 8.\" Software is furnished to do so, subject to the following conditions: 9.\" 10.\" The above copyright notice and this permission notice shall be included in 11.\" all copies or substantial portions of the Software. 12.\" 13.\" The names and trademarks of copyright holders may not be used in 14.\" advertising or publicity pertaining to the software without specific 15.\" prior permission. Title to copyright in this software and any associated 16.\" documentation will at all times remain with the copyright holders. 17.\" 18.\" THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19.\" IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20.\" FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21.\" AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22.\" LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 23.\" FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 24.\" DEALINGS IN THE SOFTWARE. 25.\" 26.\" $FreeBSD: /repoman/r/ncvs/src/share/man/man4/multicast.4,v 1.1 2003/10/17 15:12:01 bmah Exp $ 27.\" $DragonFly: src/share/man/man4/multicast.4,v 1.1 2003/10/18 21:40:41 hmp Exp $ 28.\" 29.Dd September 4, 2003 30.Dt MULTICAST 4 31.Os 32.\" 33.Sh NAME 34.Nm multicast 35.Nd Multicast Routing 36.\" 37.Sh SYNOPSIS 38.Cd "options MROUTING" 39.Pp 40.In sys/types.h 41.In sys/socket.h 42.In netinet/in.h 43.In netinet/ip_mroute.h 44.In netinet6/ip6_mroute.h 45.Ft int 46.Fn getsockopt "int s" IPPROTO_IP MRT_INIT "void *optval" "socklen_t *optlen" 47.Ft int 48.Fn setsockopt "int s" IPPROTO_IP MRT_INIT "const void *optval" "socklen_t optlen" 49.Ft int 50.Fn getsockopt "int s" IPPROTO_IPV6 MRT6_INIT "void *optval" "socklen_t *optlen" 51.Ft int 52.Fn setsockopt "int s" IPPROTO_IPV6 MRT6_INIT "const void *optval" "socklen_t optlen" 53.Sh DESCRIPTION 54.Tn "Multicast routing" 55is used to efficiently propagate data 56packets to a set of multicast listeners in multipoint networks. 57If unicast is used to replicate the data to all listeners, 58then some of the network links may carry multiple copies of the same 59data packets. 60With multicast routing, the overhead is reduced to one copy 61(at most) per network link. 62.Pp 63All multicast-capable routers must run a common multicast routing 64protocol. 65The Distance Vector Multicast Routing Protocol (DVMRP) 66was the first developed multicast routing protocol. 67Later, other protocols such as Multicast Extensions to OSPF (MOSPF), 68Core Based Trees (CBT), 69Protocol Independent Multicast - Sparse Mode (PIM-SM), 70and Protocol Independent Multicast - Dense Mode (PIM-DM) 71were developed as well. 72.Pp 73To start multicast routing, 74the user must enable multicast forwarding in the kernel 75(see 76.Sx SYNOPSIS 77about the kernel configuration options), 78and must run a multicast routing capable user-level process. 79From developer's point of view, 80the programming guide described in the 81.Sx "Programming Guide" 82section should be used to control the multicast forwarding in the kernel. 83.\" 84.Ss Programming Guide 85This section provides information about the basic multicast routing API. 86The so-called 87.Dq advanced multicast API 88is described in the 89.Sx "Advanced Multicast API Programming Guide" 90section. 91.Pp 92First, a multicast routing socket must be open. 93That socket would be used 94to control the multicast forwarding in the kernel. 95Note that most operations below require certain privilege 96(i.e., root privilege): 97.Pp 98.Bd -literal 99/* IPv4 */ 100int mrouter_s4; 101mrouter_s4 = socket(AF_INET, SOCK_RAW, IPPROTO_IGMP); 102.Ed 103.Pp 104.Bd -literal 105int mrouter_s6; 106mrouter_s6 = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); 107.Ed 108.Pp 109Note that if the router needs to open an IGMP or ICMPv6 socket 110(in case of IPv4 and IPv6 respectively) 111for sending or receiving of IGMP or MLD multicast group membership messages, 112then the same mrouter_s4 or mrouter_s6 sockets should be used 113for sending and receiving respectively IGMP or MLD messages. 114In case of BSD-derived kernel, it may be possible to open separate sockets 115for IGMP or MLD messages only. 116However, some other kernels (e.g., Linux) require that the multicast 117routing socket must be used for sending and receiving of IGMP or MLD 118messages. 119Therefore, for portability reason the multicast 120routing socket should be reused for IGMP and MLD messages as well. 121.Pp 122After the multicast routing socket is open, it can be used to enable 123or disable multicast forwarding in the kernel: 124.Bd -literal 125/* IPv4 */ 126int v = 1; /* 1 to enable, or 0 to disable */ 127setsockopt(mrouter_s4, IPPROTO_IP, MRT_INIT, (void *)&v, sizeof(v)); 128.Ed 129.Pp 130.Bd -literal 131/* IPv6 */ 132int v = 1; /* 1 to enable, or 0 to disable */ 133setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_INIT, (void *)&v, sizeof(v)); 134\&... 135/* If necessary, filter all ICMPv6 messages */ 136struct icmp6_filter filter; 137ICMP6_FILTER_SETBLOCKALL(&filter); 138setsockopt(mrouter_s6, IPPROTO_ICMPV6, ICMP6_FILTER, (void *)&filter, 139 sizeof(filter)); 140.Ed 141.Pp 142After multicast forwarding is enabled, the multicast routing socket 143can be used to enable PIM processing in the kernel if we are running PIM-SM or 144PIM-DM 145(see 146.Xr pim 4 ) . 147.Pp 148For each network interface (e.g., physical or a virtual tunnel) 149that would be used for multicast forwarding, a corresponding 150multicast interface must be added to the kernel: 151.Bd -literal 152/* IPv4 */ 153struct vifctl vc; 154memset(&vc, 0, sizeof(vc)); 155/* Assign all vifctl fields as appropriate */ 156vc.vifc_vifi = vif_index; 157vc.vifc_flags = vif_flags; 158vc.vifc_threshold = min_ttl_threshold; 159vc.vifc_rate_limit = max_rate_limit; 160memcpy(&vc.vifc_lcl_addr, &vif_local_address, sizeof(vc.vifc_lcl_addr)); 161if (vc.vifc_flags & VIFF_TUNNEL) 162 memcpy(&vc.vifc_rmt_addr, &vif_remote_address, 163 sizeof(vc.vifc_rmt_addr)); 164setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_VIF, (void *)&vc, 165 sizeof(vc)); 166.Ed 167.Pp 168The 169.Dq vif_index 170must be unique per vif. 171The 172.Dq vif_flags 173contains the 174.Dq VIFF_* 175flags as defined in <netinet/ip_mroute.h>. 176The 177.Dq min_ttl_threshold 178contains the minimum TTL a multicast data packet must have to be 179forwarded on that vif. 180Typically, it would have value of 1. 181The 182.Dq max_rate_limit 183contains the maximum rate (in bits/s) of the multicast data packets forwarded 184on that vif. 185Value of 0 means no limit. 186The 187.Dq vif_local_address 188contains the local IP address of the corresponding local interface. 189The 190.Dq vif_remote_address 191contains the remote IP address in case of DVMRP multicast tunnels. 192.Pp 193.Bd -literal 194/* IPv6 */ 195struct mif6ctl mc; 196memset(&mc, 0, sizeof(mc)); 197/* Assign all mif6ctl fields as appropriate */ 198mc.mif6c_mifi = mif_index; 199mc.mif6c_flags = mif_flags; 200mc.mif6c_pifi = pif_index; 201setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_ADD_MIF, (void *)&mc, 202 sizeof(mc)); 203.Ed 204.Pp 205The 206.Dq mif_index 207must be unique per vif. 208The 209.Dq mif_flags 210contains the 211.Dq MIFF_* 212flags as defined in <netinet6/ip6_mroute.h>. 213The 214.Dq pif_index 215is the physical interface index of the corresponding local interface. 216.Pp 217A multicast interface is deleted by: 218.Bd -literal 219/* IPv4 */ 220vifi_t vifi = vif_index; 221setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_VIF, (void *)&vifi, 222 sizeof(vifi)); 223.Ed 224.Pp 225.Bd -literal 226/* IPv6 */ 227mifi_t mifi = mif_index; 228setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_DEL_MIF, (void *)&mifi, 229 sizeof(mifi)); 230.Ed 231.Pp 232After the multicast forwarding is enabled, and the multicast virtual 233interfaces are 234added, the kernel may deliver upcall messages (also called signals 235later in this text) on the multicast routing socket that was open 236earlier with 237.Dq MRT_INIT 238or 239.Dq MRT6_INIT . 240The IPv4 upcalls have 241.Dq struct igmpmsg 242header (see <netinet/ip_mroute.h>) with field 243.Dq im_mbz 244set to zero. 245Note that this header follows the structure of 246.Dq struct ip 247with the protocol field 248.Dq ip_p 249set to zero. 250The IPv6 upcalls have 251.Dq struct mrt6msg 252header (see <netinet6/ip6_mroute.h>) with field 253.Dq im6_mbz 254set to zero. 255Note that this header follows the structure of 256.Dq struct ip6_hdr 257with the next header field 258.Dq ip6_nxt 259set to zero. 260.Pp 261The upcall header contains field 262.Dq im_msgtype 263and 264.Dq im6_msgtype 265with the type of the upcall 266.Dq IGMPMSG_* 267and 268.Dq MRT6MSG_* 269for IPv4 and IPv6 respectively. 270The values of the rest of the upcall header fields 271and the body of the upcall message depend on the particular upcall type. 272.Pp 273If the upcall message type is 274.Dq IGMPMSG_NOCACHE 275or 276.Dq MRT6MSG_NOCACHE , 277this is an indication that a multicast packet has reached the multicast 278router, but the router has no forwarding state for that packet. 279Typically, the upcall would be a signal for the multicast routing 280user-level process to install the appropriate Multicast Forwarding 281Cache (MFC) entry in the kernel. 282.Pp 283A MFC entry is added by: 284.Bd -literal 285/* IPv4 */ 286struct mfcctl mc; 287memset(&mc, 0, sizeof(mc)); 288memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 289memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 290mc.mfcc_parent = iif_index; 291for (i = 0; i < maxvifs; i++) 292 mc.mfcc_ttls[i] = oifs_ttl[i]; 293setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_MFC, 294 (void *)&mc, sizeof(mc)); 295.Ed 296.Pp 297.Bd -literal 298/* IPv6 */ 299struct mf6cctl mc; 300memset(&mc, 0, sizeof(mc)); 301memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 302memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 303mc.mf6cc_parent = iif_index; 304for (i = 0; i < maxvifs; i++) 305 if (oifs_ttl[i] > 0) 306 IF_SET(i, &mc.mf6cc_ifset); 307setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_ADD_MFC, 308 (void *)&mc, sizeof(mc)); 309.Ed 310.Pp 311The 312.Dq source_addr 313and 314.Dq group_addr 315are the source and group address of the multicast packet (as set 316in the upcall message). 317The 318.Dq iif_index 319is the virtual interface index of the multicast interface the multicast 320packets for this specific source and group address should be received on. 321The 322.Dq oifs_ttl[] 323array contains the minimum TTL (per interface) a multicast packet 324should have to be forwarded on an outgoing interface. 325If the TTL value is zero, the corresponding interface is not included 326in the set of outgoing interfaces. 327Note that in case of IPv6 only the set of outgoing interfaces can 328be specified. 329.Pp 330A MFC entry is deleted by: 331.Bd -literal 332/* IPv4 */ 333struct mfcctl mc; 334memset(&mc, 0, sizeof(mc)); 335memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 336memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 337setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_MFC, 338 (void *)&mc, sizeof(mc)); 339.Ed 340.Pp 341.Bd -literal 342/* IPv6 */ 343struct mf6cctl mc; 344memset(&mc, 0, sizeof(mc)); 345memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 346memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 347setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_DEL_MFC, 348 (void *)&mc, sizeof(mc)); 349.Ed 350.Pp 351The following method can be used to get various statistics per 352installed MFC entry in the kernel (e.g., the number of forwarded 353packets per source and group address): 354.Bd -literal 355/* IPv4 */ 356struct sioc_sg_req sgreq; 357memset(&sgreq, 0, sizeof(sgreq)); 358memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 359memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 360ioctl(mrouter_s4, SIOCGETSGCNT, &sgreq); 361.Ed 362.Pp 363.Bd -literal 364/* IPv6 */ 365struct sioc_sg_req6 sgreq; 366memset(&sgreq, 0, sizeof(sgreq)); 367memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 368memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 369ioctl(mrouter_s6, SIOCGETSGCNT_IN6, &sgreq); 370.Ed 371.Pp 372The following method can be used to get various statistics per 373multicast virtual interface in the kernel (e.g., the number of forwarded 374packets per interface): 375.Bd -literal 376/* IPv4 */ 377struct sioc_vif_req vreq; 378memset(&vreq, 0, sizeof(vreq)); 379vreq.vifi = vif_index; 380ioctl(mrouter_s4, SIOCGETVIFCNT, &vreq); 381.Ed 382.Pp 383.Bd -literal 384/* IPv6 */ 385struct sioc_mif_req6 mreq; 386memset(&mreq, 0, sizeof(mreq)); 387mreq.mifi = vif_index; 388ioctl(mrouter_s6, SIOCGETMIFCNT_IN6, &mreq); 389.Ed 390.Pp 391.Ss Advanced Multicast API Programming Guide 392If we want to add new features in the kernel, it becomes difficult 393to preserve backward compatibility (binary and API), 394and at the same time to allow user-level processes to take advantage of 395the new features (if the kernel supports them). 396.Pp 397One of the mechanisms that allows us to preserve the backward 398compatibility is a sort of negotiation 399between the user-level process and the kernel: 400.Bl -enum 401.It 402The user-level process tries to enable in the kernel the set of new 403features (and the corresponding API) it would like to use. 404.It 405The kernel returns the (sub)set of features it knows about 406and is willing to be enabled. 407.It 408The user-level process uses only that set of features 409the kernel has agreed on. 410.El 411.\" 412.Pp 413To support backward compatibility, if the user-level process doesn't 414ask for any new features, the kernel defaults to the basic 415multicast API (see the 416.Sx "Programming Guide" 417section). 418.\" XXX: edit as appropriate after the advanced multicast API is 419.\" supported under IPv6 420Currently, the advanced multicast API exists only for IPv4; 421in the future there will be IPv6 support as well. 422.Pp 423Below is a summary of the expandable API solution. 424Note that all new options and structures are defined 425in <netinet/ip_mroute.h> and <netinet6/ip6_mroute.h>, 426unless stated otherwise. 427.Pp 428The user-level process uses new get/setsockopt() options to 429perform the API features negotiation with the kernel. 430This negotiation must be performed right after the multicast routing 431socket is open. 432The set of desired/allowed features is stored in a bitset 433(currently, in uint32_t; i.e., maximum of 32 new features). 434The new get/setsockopt() options are 435.Dq MRT_API_SUPPORT 436and 437.Dq MRT_API_CONFIG . 438Example: 439.Bd -literal 440uint32_t v; 441getsockopt(sock, IPPROTO_IP, MRT_API_SUPPORT, (void *)&v, sizeof(v)); 442.Ed 443.Pp 444would set in 445.Dq v 446the pre-defined bits that the kernel API supports. 447The eight least significant bits in uint32_t are same as the 448eight possible flags 449.Dq MRT_MFC_FLAGS_* 450that can be used in 451.Dq mfcc_flags 452as part of the new definition of 453.Dq struct mfcctl 454(see below about those flags), which leaves 24 flags for other new features. 455The value returned by getsockopt(MRT_API_SUPPORT) is read-only; in other 456words, setsockopt(MRT_API_SUPPORT) would fail. 457.Pp 458To modify the API, and to set some specific feature in the kernel, then: 459.Bd -literal 460uint32_t v = MRT_MFC_FLAGS_DISABLE_WRONGVIF; 461if (setsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)) 462 != 0) { 463 return (ERROR); 464} 465if (v & MRT_MFC_FLAGS_DISABLE_WRONGVIF) 466 return (OK); /* Success */ 467else 468 return (ERROR); 469.Ed 470.Pp 471In other words, when setsockopt(MRT_API_CONFIG) is called, the 472argument to it specifies the desired set of features to 473be enabled in the API and the kernel. 474The return value in 475.Dq v 476is the actual (sub)set of features that were enabled in the kernel. 477To obtain later the same set of features that were enabled, then: 478.Bd -literal 479getsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)); 480.Ed 481.Pp 482The set of enabled features is global. 483In other words, setsockopt(MRT_API_CONFIG) 484should be called right after setsockopt(MRT_INIT). 485.Pp 486Currently, the following set of new features is defined: 487.Bd -literal 488#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 489#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 490#define MRT_MFC_RP (1 << 8) /* enable RP address */ 491#define MRT_MFC_BW_UPCALL (1 << 9) /* enable bw upcalls */ 492.Ed 493.\" .Pp 494.\" In the future there might be: 495.\" .Bd -literal 496.\" #define MRT_MFC_GROUP_SPECIFIC (1 << 10) /* allow (*,G) MFC entries */ 497.\" .Ed 498.\" .Pp 499.\" to allow (*,G) MFC entries (i.e., group-specific entries) in the kernel. 500.\" For now this is left-out until it is clear whether 501.\" (*,G) MFC support is the preferred solution instead of something more generic 502.\" solution for example. 503.\" 504.\" 2. The newly defined struct mfcctl2. 505.\" 506.Pp 507The advanced multicast API uses a newly defined 508.Dq struct mfcctl2 509instead of the traditional 510.Dq struct mfcctl . 511The original 512.Dq struct mfcctl 513is kept as is. 514The new 515.Dq struct mfcctl2 516is: 517.Bd -literal 518/* 519 * The new argument structure for MRT_ADD_MFC and MRT_DEL_MFC overlays 520 * and extends the old struct mfcctl. 521 */ 522struct mfcctl2 { 523 /* the mfcctl fields */ 524 struct in_addr mfcc_origin; /* ip origin of mcasts */ 525 struct in_addr mfcc_mcastgrp; /* multicast group associated*/ 526 vifi_t mfcc_parent; /* incoming vif */ 527 u_char mfcc_ttls[MAXVIFS];/* forwarding ttls on vifs */ 528 529 /* extension fields */ 530 uint8_t mfcc_flags[MAXVIFS];/* the MRT_MFC_FLAGS_* flags*/ 531 struct in_addr mfcc_rp; /* the RP address */ 532}; 533.Ed 534.Pp 535The new fields are 536.Dq mfcc_flags[MAXVIFS] 537and 538.Dq mfcc_rp . 539Note that for compatibility reasons they are added at the end. 540.Pp 541The 542.Dq mfcc_flags[MAXVIFS] 543field is used to set various flags per 544interface per (S,G) entry. 545Currently, the defined flags are: 546.Bd -literal 547#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 548#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 549.Ed 550.Pp 551The 552.Dq MRT_MFC_FLAGS_DISABLE_WRONGVIF 553flag is used to explicitly disable the 554.Dq IGMPMSG_WRONGVIF 555kernel signal at the (S,G) granularity if a multicast data packet 556arrives on the wrong interface. 557Usually, this signal is used to 558complete the shortest-path switch in case of PIM-SM multicast routing, 559or to trigger a PIM assert message. 560However, it should not be delivered for interfaces that are not in 561the outgoing interface set, and that are not expecting to 562become an incoming interface. 563Hence, if the 564.Dq MRT_MFC_FLAGS_DISABLE_WRONGVIF 565flag is set for some of the 566interfaces, then a data packet that arrives on that interface for 567that MFC entry will NOT trigger a WRONGVIF signal. 568If that flag is not set, then a signal is triggered (the default action). 569.Pp 570The 571.Dq MRT_MFC_FLAGS_BORDER_VIF 572flag is used to specify whether the Border-bit in PIM 573Register messages should be set (in case when the Register encapsulation 574is performed inside the kernel). 575If it is set for the special PIM Register kernel virtual interface 576(see 577.Xr pim 4 ) , 578the Border-bit in the Register messages sent to the RP will be set. 579.Pp 580The remaining six bits are reserved for future usage. 581.Pp 582The 583.Dq mfcc_rp 584field is used to specify the RP address (in case of PIM-SM multicast routing) 585for a multicast 586group G if we want to perform kernel-level PIM Register encapsulation. 587The 588.Dq mfcc_rp 589field is used only if the 590.Dq MRT_MFC_RP 591advanced API flag/capability has been successfully set by 592setsockopt(MRT_API_CONFIG). 593.Pp 594.\" 595.\" 3. Kernel-level PIM Register encapsulation 596.\" 597If the 598.Dq MRT_MFC_RP 599flag was successfully set by 600setsockopt(MRT_API_CONFIG), then the kernel will attempt to perform 601the PIM Register encapsulation itself instead of sending the 602multicast data packets to user level (inside IGMPMSG_WHOLEPKT 603upcalls) for user-level encapsulation. 604The RP address would be taken from the 605.Dq mfcc_rp 606field 607inside the new 608.Dq struct mfcctl2 . 609However, even if the 610.Dq MRT_MFC_RP 611flag was successfully set, if the 612.Dq mfcc_rp 613field was set to 614.Dq INADDR_ANY , 615then the 616kernel will still deliver an IGMPMSG_WHOLEPKT upcall with the 617multicast data packet to the user-level process. 618.Pp 619In addition, if the multicast data packet is too large to fit within 620a single IP packet after the PIM Register encapsulation (e.g., if 621its size was on the order of 65500 bytes), the data packet will be 622fragmented, and then each of the fragments will be encapsulated 623separately. 624Note that typically a multicast data packet can be that 625large only if it was originated locally from the same hosts that 626performs the encapsulation; otherwise the transmission of the 627multicast data packet over Ethernet for example would have 628fragmented it into much smaller pieces. 629.\" 630.\" Note that if this code is ported to IPv6, we may need the kernel to 631.\" perform MTU discovery to the RP, and keep those discoveries inside 632.\" the kernel so the encapsulating router may send back ICMP 633.\" Fragmentation Required if the size of the multicast data packet is 634.\" too large (see "Encapsulating data packets in the Register Tunnel" 635.\" in Section 4.4.1 in the PIM-SM spec 636.\" draft-ietf-pim-sm-v2-new-05.{txt,ps}). 637.\" For IPv4 we may be able to get away without it, but for IPv6 we need 638.\" that. 639.\" 640.\" 4. Mechanism for "multicast bandwidth monitoring and upcalls". 641.\" 642.Pp 643Typically, a multicast routing user-level process would need to know the 644forwarding bandwidth for some data flow. 645For example, the multicast routing process may want to timeout idle MFC 646entries, or in case of PIM-SM it can initiate (S,G) shortest-path switch if 647the bandwidth rate is above a threshold for example. 648.Pp 649The original solution for measuring the bandwidth of a dataflow was 650that a user-level process would periodically 651query the kernel about the number of forwarded packets/bytes per 652(S,G), and then based on those numbers it would estimate whether a source 653has been idle, or whether the source's transmission bandwidth is above a 654threshold. 655That solution is far from being scalable, hence the need for a new 656mechanism for bandwidth monitoring. 657.Pp 658Below is a description of the bandwidth monitoring mechanism. 659.Bl -bullet 660.It 661If the bandwidth of a data flow satisfies some pre-defined filter, 662the kernel delivers an upcall on the multicast routing socket 663to the multicast routing process that has installed that filter. 664.It 665The bandwidth-upcall filters are installed per (S,G). There can be 666more than one filter per (S,G). 667.It 668Instead of supporting all possible comparison operations 669(i.e., < <= == != > >= ), there is support only for the 670<= and >= operations, 671because this makes the kernel-level implementation simpler, 672and because practically we need only those two. 673Further, the missing operations can be simulated by secondary 674user-level filtering of those <= and >= filters. 675For example, to simulate !=, then we need to install filter 676.Dq bw <= 0xffffffff , 677and after an 678upcall is received, we need to check whether 679.Dq measured_bw != expected_bw . 680.It 681The bandwidth-upcall mechanism is enabled by 682setsockopt(MRT_API_CONFIG) for the MRT_MFC_BW_UPCALL flag. 683.It 684The bandwidth-upcall filters are added/deleted by the new 685setsockopt(MRT_ADD_BW_UPCALL) and setsockopt(MRT_DEL_BW_UPCALL) 686respectively (with the appropriate 687.Dq struct bw_upcall 688argument of course). 689.El 690.Pp 691From application point of view, a developer needs to know about 692the following: 693.Bd -literal 694/* 695 * Structure for installing or delivering an upcall if the 696 * measured bandwidth is above or below a threshold. 697 * 698 * User programs (e.g. daemons) may have a need to know when the 699 * bandwidth used by some data flow is above or below some threshold. 700 * This interface allows the userland to specify the threshold (in 701 * bytes and/or packets) and the measurement interval. Flows are 702 * all packet with the same source and destination IP address. 703 * At the moment the code is only used for multicast destinations 704 * but there is nothing that prevents its use for unicast. 705 * 706 * The measurement interval cannot be shorter than some Tmin (currently, 3s). 707 * The threshold is set in packets and/or bytes per_interval. 708 * 709 * Measurement works as follows: 710 * 711 * For >= measurements: 712 * The first packet marks the start of a measurement interval. 713 * During an interval we count packets and bytes, and when we 714 * pass the threshold we deliver an upcall and we are done. 715 * The first packet after the end of the interval resets the 716 * count and restarts the measurement. 717 * 718 * For <= measurement: 719 * We start a timer to fire at the end of the interval, and 720 * then for each incoming packet we count packets and bytes. 721 * When the timer fires, we compare the value with the threshold, 722 * schedule an upcall if we are below, and restart the measurement 723 * (reschedule timer and zero counters). 724 */ 725 726struct bw_data { 727 struct timeval b_time; 728 uint64_t b_packets; 729 uint64_t b_bytes; 730}; 731 732struct bw_upcall { 733 struct in_addr bu_src; /* source address */ 734 struct in_addr bu_dst; /* destination address */ 735 uint32_t bu_flags; /* misc flags (see below) */ 736#define BW_UPCALL_UNIT_PACKETS (1 << 0) /* threshold (in packets) */ 737#define BW_UPCALL_UNIT_BYTES (1 << 1) /* threshold (in bytes) */ 738#define BW_UPCALL_GEQ (1 << 2) /* upcall if bw >= threshold */ 739#define BW_UPCALL_LEQ (1 << 3) /* upcall if bw <= threshold */ 740#define BW_UPCALL_DELETE_ALL (1 << 4) /* delete all upcalls for s,d*/ 741 struct bw_data bu_threshold; /* the bw threshold */ 742 struct bw_data bu_measured; /* the measured bw */ 743}; 744 745/* max. number of upcalls to deliver together */ 746#define BW_UPCALLS_MAX 128 747/* min. threshold time interval for bandwidth measurement */ 748#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_SEC 3 749#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_USEC 0 750.Ed 751.Pp 752The 753.Dq bw_upcall 754structure is used as an argument to 755setsockopt(MRT_ADD_BW_UPCALL) and setsockopt(MRT_DEL_BW_UPCALL). 756Each setsockopt(MRT_ADD_BW_UPCALL) installs a filter in the kernel 757for the source and destination address in the 758.Dq bw_upcall 759argument, 760and that filter will trigger an upcall according to the following 761pseudo-algorithm: 762.Bd -literal 763 if (bw_upcall_oper IS ">=") { 764 if (((bw_upcall_unit & PACKETS == PACKETS) && 765 (measured_packets >= threshold_packets)) || 766 ((bw_upcall_unit & BYTES == BYTES) && 767 (measured_bytes >= threshold_bytes))) 768 SEND_UPCALL("measured bandwidth is >= threshold"); 769 } 770 if (bw_upcall_oper IS "<=" && measured_interval >= threshold_interval) { 771 if (((bw_upcall_unit & PACKETS == PACKETS) && 772 (measured_packets <= threshold_packets)) || 773 ((bw_upcall_unit & BYTES == BYTES) && 774 (measured_bytes <= threshold_bytes))) 775 SEND_UPCALL("measured bandwidth is <= threshold"); 776 } 777.Ed 778.Pp 779In the same 780.Dq bw_upcall 781the unit can be specified in both BYTES and PACKETS. 782However, the GEQ and LEQ flags are mutually exclusive. 783.Pp 784Basically, an upcall is delivered if the measured bandwidth is >= or 785<= the threshold bandwidth (within the specified measurement 786interval). 787For practical reasons, the smallest value for the measurement 788interval is 3 seconds. 789If smaller values are allowed, then the bandwidth 790estimation may be less accurate, or the potentially very high frequency 791of the generated upcalls may introduce too much overhead. 792For the >= operation, the answer may be known before the end of 793.Dq threshold_interval , 794therefore the upcall may be delivered earlier. 795For the <= operation however, we must wait 796until the threshold interval has expired to know the answer. 797.Pp 798Example of usage: 799.Bd -literal 800struct bw_upcall bw_upcall; 801/* Assign all bw_upcall fields as appropriate */ 802memset(&bw_upcall, 0, sizeof(bw_upcall)); 803memcpy(&bw_upcall.bu_src, &source, sizeof(bw_upcall.bu_src)); 804memcpy(&bw_upcall.bu_dst, &group, sizeof(bw_upcall.bu_dst)); 805bw_upcall.bu_threshold.b_data = threshold_interval; 806bw_upcall.bu_threshold.b_packets = threshold_packets; 807bw_upcall.bu_threshold.b_bytes = threshold_bytes; 808if (is_threshold_in_packets) 809 bw_upcall.bu_flags |= BW_UPCALL_UNIT_PACKETS; 810if (is_threshold_in_bytes) 811 bw_upcall.bu_flags |= BW_UPCALL_UNIT_BYTES; 812do { 813 if (is_geq_upcall) { 814 bw_upcall.bu_flags |= BW_UPCALL_GEQ; 815 break; 816 } 817 if (is_leq_upcall) { 818 bw_upcall.bu_flags |= BW_UPCALL_LEQ; 819 break; 820 } 821 return (ERROR); 822} while (0); 823setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_BW_UPCALL, 824 (void *)&bw_upcall, sizeof(bw_upcall)); 825.Ed 826.Pp 827To delete a single filter, then use MRT_DEL_BW_UPCALL, 828and the fields of bw_upcall must be set 829exactly same as when MRT_ADD_BW_UPCALL was called. 830.Pp 831To delete all bandwidth filters for a given (S,G), then 832only the 833.Dq bu_src 834and 835.Dq bu_dst 836fields in 837.Dq struct bw_upcall 838need to be set, and then just set only the 839.Dq BW_UPCALL_DELETE_ALL 840flag inside field 841.Dq bw_upcall.bu_flags . 842.Pp 843The bandwidth upcalls are received by aggregating them in the new upcall 844message: 845.Bd -literal 846#define IGMPMSG_BW_UPCALL 4 /* BW monitoring upcall */ 847.Ed 848.Pp 849This message is an array of 850.Dq struct bw_upcall 851elements (up to BW_UPCALLS_MAX = 128). 852The upcalls are 853delivered when there are 128 pending upcalls, or when 1 second has 854expired since the previous upcall (whichever comes first). 855In an 856.Dq struct upcall 857element, the 858.Dq bu_measured 859field is filled-in to 860indicate the particular measured values. 861However, because of the way 862the particular intervals are measured, the user should be careful how 863bu_measured.b_time is used. 864For example, if the 865filter is installed to trigger an upcall if the number of packets 866is >= 1, then 867.Dq bu_measured 868may have a value of zero in the upcalls after the 869first one, because the measured interval for >= filters is 870.Dq clocked 871by the forwarded packets. 872Hence, this upcall mechanism should not be used for measuring 873the exact value of the bandwidth of the forwarded data. 874To measure the exact bandwidth, the user would need to 875get the forwarded packets statistics with the ioctl(SIOCGETSGCNT) 876mechanism 877(see the 878.Sx Programming Guide 879section) . 880.Pp 881Note that the upcalls for a filter are delivered until the specific 882filter is deleted, but no more frequently than once per 883.Dq bu_threshold.b_time . 884For example, if the filter is specified to 885deliver a signal if bw >= 1 packet, the first packet will trigger a 886signal, but the next upcall will be triggered no earlier than 887.Dq bu_threshold.b_time 888after the previous upcall. 889.Pp 890.\" 891.Sh SEE ALSO 892.Xr getsockopt 2 , 893.Xr recvfrom 2 , 894.Xr recvmsg 2 , 895.Xr setsockopt 2 , 896.Xr socket 2 , 897.Xr icmp6 4 , 898.Xr inet 4 , 899.Xr inet6 4 , 900.Xr intro 4 , 901.Xr ip 4 , 902.Xr ip6 4 , 903.Xr pim 4 904.\" 905.Pp 906.Sh AUTHORS 907The original multicast code was written by David Waitzman (BBN Labs), 908and later modified by the following individuals: 909Steve Deering (Stanford), Mark J. Steiglitz (Stanford), 910Van Jacobson (LBL), Ajit Thyagarajan (PARC), 911Bill Fenner (PARC). 912The IPv6 multicast support was implemented by the KAME project 913(http://www.kame.net), and was based on the IPv4 multicast code. 914The advanced multicast API and the multicast bandwidth 915monitoring were implemented by Pavlin Radoslavov (ICSI) 916in collaboration with Chris Brown (NextHop). 917.Pp 918This manual page was written by Pavlin Radoslavov (ICSI). 919