1.\" Copyright (c) 2001-2003 International Computer Science Institute 2.\" 3.\" Permission is hereby granted, free of charge, to any person obtaining a 4.\" copy of this software and associated documentation files (the "Software"), 5.\" to deal in the Software without restriction, including without limitation 6.\" the rights to use, copy, modify, merge, publish, distribute, sublicense, 7.\" and/or sell copies of the Software, and to permit persons to whom the 8.\" Software is furnished to do so, subject to the following conditions: 9.\" 10.\" The above copyright notice and this permission notice shall be included in 11.\" all copies or substantial portions of the Software. 12.\" 13.\" The names and trademarks of copyright holders may not be used in 14.\" advertising or publicity pertaining to the software without specific 15.\" prior permission. Title to copyright in this software and any associated 16.\" documentation will at all times remain with the copyright holders. 17.\" 18.\" THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19.\" IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20.\" FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21.\" AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22.\" LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 23.\" FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 24.\" DEALINGS IN THE SOFTWARE. 25.\" 26.\" $FreeBSD: /repoman/r/ncvs/src/share/man/man4/multicast.4,v 1.1 2003/10/17 15:12:01 bmah Exp $ 27.\" $DragonFly: src/share/man/man4/multicast.4,v 1.7 2008/05/02 02:05:05 swildner Exp $ 28.\" 29.Dd September 4, 2003 30.Dt MULTICAST 4 31.Os 32.\" 33.Sh NAME 34.Nm multicast 35.Nd Multicast Routing 36.\" 37.Sh SYNOPSIS 38.Cd "options MROUTING" 39.Pp 40.In sys/types.h 41.In sys/socket.h 42.In netinet/in.h 43.In net/ip_mroute/ip_mroute.h 44.In netinet6/ip6_mroute.h 45.Ft int 46.Fn getsockopt "int s" IPPROTO_IP MRT_INIT "void *optval" "socklen_t *optlen" 47.Ft int 48.Fn setsockopt "int s" IPPROTO_IP MRT_INIT "const void *optval" "socklen_t optlen" 49.Ft int 50.Fn getsockopt "int s" IPPROTO_IPV6 MRT6_INIT "void *optval" "socklen_t *optlen" 51.Ft int 52.Fn setsockopt "int s" IPPROTO_IPV6 MRT6_INIT "const void *optval" "socklen_t optlen" 53.Sh DESCRIPTION 54.Tn "Multicast routing" 55is used to efficiently propagate data 56packets to a set of multicast listeners in multipoint networks. 57If unicast is used to replicate the data to all listeners, 58then some of the network links may carry multiple copies of the same 59data packets. 60With multicast routing, the overhead is reduced to one copy 61(at most) per network link. 62.Pp 63All multicast-capable routers must run a common multicast routing 64protocol. 65The Distance Vector Multicast Routing Protocol (DVMRP) 66was the first developed multicast routing protocol. 67Later, other protocols such as Multicast Extensions to OSPF (MOSPF), 68Core Based Trees (CBT), 69Protocol Independent Multicast - Sparse Mode (PIM-SM), 70and Protocol Independent Multicast - Dense Mode (PIM-DM) 71were developed as well. 72.Pp 73To start multicast routing, 74the user must enable multicast forwarding in the kernel 75(see 76.Sx SYNOPSIS 77about the kernel configuration options), 78and must run a multicast routing capable user-level process. 79From developer's point of view, 80the programming guide described in the 81.Sx "Programming Guide" 82section should be used to control the multicast forwarding in the kernel. 83.\" 84.Ss Programming Guide 85This section provides information about the basic multicast routing API. 86The so-called 87.Dq advanced multicast API 88is described in the 89.Sx "Advanced Multicast API Programming Guide" 90section. 91.Pp 92First, a multicast routing socket must be open. 93That socket would be used 94to control the multicast forwarding in the kernel. 95Note that most operations below require certain privilege 96(i.e., root privilege): 97.Bd -literal 98/* IPv4 */ 99int mrouter_s4; 100mrouter_s4 = socket(AF_INET, SOCK_RAW, IPPROTO_IGMP); 101.Ed 102.Bd -literal 103int mrouter_s6; 104mrouter_s6 = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); 105.Ed 106.Pp 107Note that if the router needs to open an IGMP or ICMPv6 socket 108(in case of IPv4 and IPv6 respectively) 109for sending or receiving of IGMP or MLD multicast group membership messages, 110then the same mrouter_s4 or mrouter_s6 sockets should be used 111for sending and receiving respectively IGMP or MLD messages. 112In case of BSD-derived kernel, it may be possible to open separate sockets 113for IGMP or MLD messages only. 114However, some other kernels (e.g., Linux) require that the multicast 115routing socket must be used for sending and receiving of IGMP or MLD 116messages. 117Therefore, for portability reason the multicast 118routing socket should be reused for IGMP and MLD messages as well. 119.Pp 120After the multicast routing socket is open, it can be used to enable 121or disable multicast forwarding in the kernel: 122.Bd -literal 123/* IPv4 */ 124int v = 1; /* 1 to enable, or 0 to disable */ 125setsockopt(mrouter_s4, IPPROTO_IP, MRT_INIT, (void *)&v, sizeof(v)); 126.Ed 127.Bd -literal 128/* IPv6 */ 129int v = 1; /* 1 to enable, or 0 to disable */ 130setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_INIT, (void *)&v, sizeof(v)); 131\&... 132/* If necessary, filter all ICMPv6 messages */ 133struct icmp6_filter filter; 134ICMP6_FILTER_SETBLOCKALL(&filter); 135setsockopt(mrouter_s6, IPPROTO_ICMPV6, ICMP6_FILTER, (void *)&filter, 136 sizeof(filter)); 137.Ed 138.Pp 139After multicast forwarding is enabled, the multicast routing socket 140can be used to enable PIM processing in the kernel if we are running PIM-SM or 141PIM-DM 142(see 143.Xr pim 4 ) . 144.Pp 145For each network interface (e.g., physical or a virtual tunnel) 146that would be used for multicast forwarding, a corresponding 147multicast interface must be added to the kernel: 148.Bd -literal 149/* IPv4 */ 150struct vifctl vc; 151memset(&vc, 0, sizeof(vc)); 152/* Assign all vifctl fields as appropriate */ 153vc.vifc_vifi = vif_index; 154vc.vifc_flags = vif_flags; 155vc.vifc_threshold = min_ttl_threshold; 156vc.vifc_rate_limit = max_rate_limit; 157memcpy(&vc.vifc_lcl_addr, &vif_local_address, sizeof(vc.vifc_lcl_addr)); 158if (vc.vifc_flags & VIFF_TUNNEL) 159 memcpy(&vc.vifc_rmt_addr, &vif_remote_address, 160 sizeof(vc.vifc_rmt_addr)); 161setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_VIF, (void *)&vc, 162 sizeof(vc)); 163.Ed 164.Pp 165The 166.Dq vif_index 167must be unique per vif. 168The 169.Dq vif_flags 170contains the 171.Dq VIFF_* 172flags as defined in 173.In net/ip_mroute/ip_mroute.h . 174The 175.Dq min_ttl_threshold 176contains the minimum TTL a multicast data packet must have to be 177forwarded on that vif. 178Typically, it would have value of 1. 179The 180.Dq max_rate_limit 181contains the maximum rate (in bits/s) of the multicast data packets forwarded 182on that vif. 183Value of 0 means no limit. 184The 185.Dq vif_local_address 186contains the local IP address of the corresponding local interface. 187The 188.Dq vif_remote_address 189contains the remote IP address in case of DVMRP multicast tunnels. 190.Bd -literal 191/* IPv6 */ 192struct mif6ctl mc; 193memset(&mc, 0, sizeof(mc)); 194/* Assign all mif6ctl fields as appropriate */ 195mc.mif6c_mifi = mif_index; 196mc.mif6c_flags = mif_flags; 197mc.mif6c_pifi = pif_index; 198setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_ADD_MIF, (void *)&mc, 199 sizeof(mc)); 200.Ed 201.Pp 202The 203.Dq mif_index 204must be unique per vif. 205The 206.Dq mif_flags 207contains the 208.Dq MIFF_* 209flags as defined in 210.In netinet6/ip6_mroute.h . 211The 212.Dq pif_index 213is the physical interface index of the corresponding local interface. 214.Pp 215A multicast interface is deleted by: 216.Bd -literal 217/* IPv4 */ 218vifi_t vifi = vif_index; 219setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_VIF, (void *)&vifi, 220 sizeof(vifi)); 221.Ed 222.Bd -literal 223/* IPv6 */ 224mifi_t mifi = mif_index; 225setsockopt(mrouter_s6, IPPROTO_IPV6, MRT6_DEL_MIF, (void *)&mifi, 226 sizeof(mifi)); 227.Ed 228.Pp 229After the multicast forwarding is enabled, and the multicast virtual 230interfaces are 231added, the kernel may deliver upcall messages (also called signals 232later in this text) on the multicast routing socket that was open 233earlier with 234.Dq MRT_INIT 235or 236.Dq MRT6_INIT . 237The IPv4 upcalls have 238.Dq struct igmpmsg 239header (see 240.In net/ip_mroute/ip_mroute.h ) 241with field 242.Dq im_mbz 243set to zero. 244Note that this header follows the structure of 245.Dq struct ip 246with the protocol field 247.Dq ip_p 248set to zero. 249The IPv6 upcalls have 250.Dq struct mrt6msg 251header (see 252.In netinet6/ip6_mroute.h ) 253with field 254.Dq im6_mbz 255set to zero. 256Note that this header follows the structure of 257.Dq struct ip6_hdr 258with the next header field 259.Dq ip6_nxt 260set to zero. 261.Pp 262The upcall header contains field 263.Dq im_msgtype 264and 265.Dq im6_msgtype 266with the type of the upcall 267.Dq IGMPMSG_* 268and 269.Dq MRT6MSG_* 270for IPv4 and IPv6 respectively. 271The values of the rest of the upcall header fields 272and the body of the upcall message depend on the particular upcall type. 273.Pp 274If the upcall message type is 275.Dq IGMPMSG_NOCACHE 276or 277.Dq MRT6MSG_NOCACHE , 278this is an indication that a multicast packet has reached the multicast 279router, but the router has no forwarding state for that packet. 280Typically, the upcall would be a signal for the multicast routing 281user-level process to install the appropriate Multicast Forwarding 282Cache (MFC) entry in the kernel. 283.Pp 284A MFC entry is added by: 285.Bd -literal 286/* IPv4 */ 287struct mfcctl mc; 288memset(&mc, 0, sizeof(mc)); 289memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 290memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 291mc.mfcc_parent = iif_index; 292for (i = 0; i < maxvifs; i++) 293 mc.mfcc_ttls[i] = oifs_ttl[i]; 294setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_MFC, 295 (void *)&mc, sizeof(mc)); 296.Ed 297.Bd -literal 298/* IPv6 */ 299struct mf6cctl mc; 300memset(&mc, 0, sizeof(mc)); 301memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 302memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 303mc.mf6cc_parent = iif_index; 304for (i = 0; i < maxvifs; i++) 305 if (oifs_ttl[i] > 0) 306 IF_SET(i, &mc.mf6cc_ifset); 307setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_ADD_MFC, 308 (void *)&mc, sizeof(mc)); 309.Ed 310.Pp 311The 312.Dq source_addr 313and 314.Dq group_addr 315are the source and group address of the multicast packet (as set 316in the upcall message). 317The 318.Dq iif_index 319is the virtual interface index of the multicast interface the multicast 320packets for this specific source and group address should be received on. 321The 322.Dq oifs_ttl[] 323array contains the minimum TTL (per interface) a multicast packet 324should have to be forwarded on an outgoing interface. 325If the TTL value is zero, the corresponding interface is not included 326in the set of outgoing interfaces. 327Note that in case of IPv6 only the set of outgoing interfaces can 328be specified. 329.Pp 330A MFC entry is deleted by: 331.Bd -literal 332/* IPv4 */ 333struct mfcctl mc; 334memset(&mc, 0, sizeof(mc)); 335memcpy(&mc.mfcc_origin, &source_addr, sizeof(mc.mfcc_origin)); 336memcpy(&mc.mfcc_mcastgrp, &group_addr, sizeof(mc.mfcc_mcastgrp)); 337setsockopt(mrouter_s4, IPPROTO_IP, MRT_DEL_MFC, 338 (void *)&mc, sizeof(mc)); 339.Ed 340.Bd -literal 341/* IPv6 */ 342struct mf6cctl mc; 343memset(&mc, 0, sizeof(mc)); 344memcpy(&mc.mf6cc_origin, &source_addr, sizeof(mc.mf6cc_origin)); 345memcpy(&mc.mf6cc_mcastgrp, &group_addr, sizeof(mf6cc_mcastgrp)); 346setsockopt(mrouter_s4, IPPROTO_IPV6, MRT6_DEL_MFC, 347 (void *)&mc, sizeof(mc)); 348.Ed 349.Pp 350The following method can be used to get various statistics per 351installed MFC entry in the kernel (e.g., the number of forwarded 352packets per source and group address): 353.Bd -literal 354/* IPv4 */ 355struct sioc_sg_req sgreq; 356memset(&sgreq, 0, sizeof(sgreq)); 357memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 358memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 359ioctl(mrouter_s4, SIOCGETSGCNT, &sgreq); 360.Ed 361.Bd -literal 362/* IPv6 */ 363struct sioc_sg_req6 sgreq; 364memset(&sgreq, 0, sizeof(sgreq)); 365memcpy(&sgreq.src, &source_addr, sizeof(sgreq.src)); 366memcpy(&sgreq.grp, &group_addr, sizeof(sgreq.grp)); 367ioctl(mrouter_s6, SIOCGETSGCNT_IN6, &sgreq); 368.Ed 369.Pp 370The following method can be used to get various statistics per 371multicast virtual interface in the kernel (e.g., the number of forwarded 372packets per interface): 373.Bd -literal 374/* IPv4 */ 375struct sioc_vif_req vreq; 376memset(&vreq, 0, sizeof(vreq)); 377vreq.vifi = vif_index; 378ioctl(mrouter_s4, SIOCGETVIFCNT, &vreq); 379.Ed 380.Bd -literal 381/* IPv6 */ 382struct sioc_mif_req6 mreq; 383memset(&mreq, 0, sizeof(mreq)); 384mreq.mifi = vif_index; 385ioctl(mrouter_s6, SIOCGETMIFCNT_IN6, &mreq); 386.Ed 387.Ss Advanced Multicast API Programming Guide 388If we want to add new features in the kernel, it becomes difficult 389to preserve backward compatibility (binary and API), 390and at the same time to allow user-level processes to take advantage of 391the new features (if the kernel supports them). 392.Pp 393One of the mechanisms that allows us to preserve the backward 394compatibility is a sort of negotiation 395between the user-level process and the kernel: 396.Bl -enum 397.It 398The user-level process tries to enable in the kernel the set of new 399features (and the corresponding API) it would like to use. 400.It 401The kernel returns the (sub)set of features it knows about 402and is willing to be enabled. 403.It 404The user-level process uses only that set of features 405the kernel has agreed on. 406.El 407.\" 408.Pp 409To support backward compatibility, if the user-level process doesn't 410ask for any new features, the kernel defaults to the basic 411multicast API (see the 412.Sx "Programming Guide" 413section). 414.\" XXX: edit as appropriate after the advanced multicast API is 415.\" supported under IPv6 416Currently, the advanced multicast API exists only for IPv4; 417in the future there will be IPv6 support as well. 418.Pp 419Below is a summary of the expandable API solution. 420Note that all new options and structures are defined 421in 422.In net/ip_mroute/ip_mroute.h 423and 424.In netinet6/ip6_mroute.h , 425unless stated otherwise. 426.Pp 427The user-level process uses new get/setsockopt() options to 428perform the API features negotiation with the kernel. 429This negotiation must be performed right after the multicast routing 430socket is open. 431The set of desired/allowed features is stored in a bitset 432(currently, in uint32_t; i.e., maximum of 32 new features). 433The new get/setsockopt() options are 434.Dq MRT_API_SUPPORT 435and 436.Dq MRT_API_CONFIG . 437Example: 438.Bd -literal 439uint32_t v; 440getsockopt(sock, IPPROTO_IP, MRT_API_SUPPORT, (void *)&v, sizeof(v)); 441.Ed 442.Pp 443would set in 444.Dq v 445the pre-defined bits that the kernel API supports. 446The eight least significant bits in uint32_t are same as the 447eight possible flags 448.Dq MRT_MFC_FLAGS_* 449that can be used in 450.Dq mfcc_flags 451as part of the new definition of 452.Dq struct mfcctl 453(see below about those flags), which leaves 24 flags for other new features. 454The value returned by getsockopt(MRT_API_SUPPORT) is read-only; in other 455words, setsockopt(MRT_API_SUPPORT) would fail. 456.Pp 457To modify the API, and to set some specific feature in the kernel, then: 458.Bd -literal 459uint32_t v = MRT_MFC_FLAGS_DISABLE_WRONGVIF; 460if (setsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)) 461 != 0) { 462 return (ERROR); 463} 464if (v & MRT_MFC_FLAGS_DISABLE_WRONGVIF) 465 return (OK); /* Success */ 466else 467 return (ERROR); 468.Ed 469.Pp 470In other words, when setsockopt(MRT_API_CONFIG) is called, the 471argument to it specifies the desired set of features to 472be enabled in the API and the kernel. 473The return value in 474.Dq v 475is the actual (sub)set of features that were enabled in the kernel. 476To obtain later the same set of features that were enabled, then: 477.Bd -literal 478getsockopt(sock, IPPROTO_IP, MRT_API_CONFIG, (void *)&v, sizeof(v)); 479.Ed 480.Pp 481The set of enabled features is global. 482In other words, setsockopt(MRT_API_CONFIG) 483should be called right after setsockopt(MRT_INIT). 484.Pp 485Currently, the following set of new features is defined: 486.Bd -literal 487#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 488#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 489#define MRT_MFC_RP (1 << 8) /* enable RP address */ 490#define MRT_MFC_BW_UPCALL (1 << 9) /* enable bw upcalls */ 491.Ed 492.\" .Pp 493.\" In the future there might be: 494.\" .Bd -literal 495.\" #define MRT_MFC_GROUP_SPECIFIC (1 << 10) /* allow (*,G) MFC entries */ 496.\" .Ed 497.\" .Pp 498.\" to allow (*,G) MFC entries (i.e., group-specific entries) in the kernel. 499.\" For now this is left-out until it is clear whether 500.\" (*,G) MFC support is the preferred solution instead of something more generic 501.\" solution for example. 502.\" 503.\" 2. The newly defined struct mfcctl2. 504.\" 505.Pp 506The advanced multicast API uses a newly defined 507.Dq struct mfcctl2 508instead of the traditional 509.Dq struct mfcctl . 510The original 511.Dq struct mfcctl 512is kept as is. 513The new 514.Dq struct mfcctl2 515is: 516.Bd -literal 517/* 518 * The new argument structure for MRT_ADD_MFC and MRT_DEL_MFC overlays 519 * and extends the old struct mfcctl. 520 */ 521struct mfcctl2 { 522 /* the mfcctl fields */ 523 struct in_addr mfcc_origin; /* ip origin of mcasts */ 524 struct in_addr mfcc_mcastgrp; /* multicast group associated*/ 525 vifi_t mfcc_parent; /* incoming vif */ 526 u_char mfcc_ttls[MAXVIFS];/* forwarding ttls on vifs */ 527 528 /* extension fields */ 529 uint8_t mfcc_flags[MAXVIFS];/* the MRT_MFC_FLAGS_* flags*/ 530 struct in_addr mfcc_rp; /* the RP address */ 531}; 532.Ed 533.Pp 534The new fields are 535.Dq mfcc_flags[MAXVIFS] 536and 537.Dq mfcc_rp . 538Note that for compatibility reasons they are added at the end. 539.Pp 540The 541.Dq mfcc_flags[MAXVIFS] 542field is used to set various flags per 543interface per (S,G) entry. 544Currently, the defined flags are: 545.Bd -literal 546#define MRT_MFC_FLAGS_DISABLE_WRONGVIF (1 << 0) /* disable WRONGVIF signals */ 547#define MRT_MFC_FLAGS_BORDER_VIF (1 << 1) /* border vif */ 548.Ed 549.Pp 550The 551.Dq MRT_MFC_FLAGS_DISABLE_WRONGVIF 552flag is used to explicitly disable the 553.Dq IGMPMSG_WRONGVIF 554kernel signal at the (S,G) granularity if a multicast data packet 555arrives on the wrong interface. 556Usually, this signal is used to 557complete the shortest-path switch in case of PIM-SM multicast routing, 558or to trigger a PIM assert message. 559However, it should not be delivered for interfaces that are not in 560the outgoing interface set, and that are not expecting to 561become an incoming interface. 562Hence, if the 563.Dq MRT_MFC_FLAGS_DISABLE_WRONGVIF 564flag is set for some of the 565interfaces, then a data packet that arrives on that interface for 566that MFC entry will NOT trigger a WRONGVIF signal. 567If that flag is not set, then a signal is triggered (the default action). 568.Pp 569The 570.Dq MRT_MFC_FLAGS_BORDER_VIF 571flag is used to specify whether the Border-bit in PIM 572Register messages should be set (in case when the Register encapsulation 573is performed inside the kernel). 574If it is set for the special PIM Register kernel virtual interface 575(see 576.Xr pim 4 ) , 577the Border-bit in the Register messages sent to the RP will be set. 578.Pp 579The remaining six bits are reserved for future usage. 580.Pp 581The 582.Dq mfcc_rp 583field is used to specify the RP address (in case of PIM-SM multicast routing) 584for a multicast 585group G if we want to perform kernel-level PIM Register encapsulation. 586The 587.Dq mfcc_rp 588field is used only if the 589.Dq MRT_MFC_RP 590advanced API flag/capability has been successfully set by 591setsockopt(MRT_API_CONFIG). 592.Pp 593.\" 594.\" 3. Kernel-level PIM Register encapsulation 595.\" 596If the 597.Dq MRT_MFC_RP 598flag was successfully set by 599setsockopt(MRT_API_CONFIG), then the kernel will attempt to perform 600the PIM Register encapsulation itself instead of sending the 601multicast data packets to user level (inside IGMPMSG_WHOLEPKT 602upcalls) for user-level encapsulation. 603The RP address would be taken from the 604.Dq mfcc_rp 605field 606inside the new 607.Dq struct mfcctl2 . 608However, even if the 609.Dq MRT_MFC_RP 610flag was successfully set, if the 611.Dq mfcc_rp 612field was set to 613.Dq INADDR_ANY , 614then the 615kernel will still deliver an IGMPMSG_WHOLEPKT upcall with the 616multicast data packet to the user-level process. 617.Pp 618In addition, if the multicast data packet is too large to fit within 619a single IP packet after the PIM Register encapsulation (e.g., if 620its size was on the order of 65500 bytes), the data packet will be 621fragmented, and then each of the fragments will be encapsulated 622separately. 623Note that typically a multicast data packet can be that 624large only if it was originated locally from the same hosts that 625performs the encapsulation; otherwise the transmission of the 626multicast data packet over Ethernet for example would have 627fragmented it into much smaller pieces. 628.\" 629.\" Note that if this code is ported to IPv6, we may need the kernel to 630.\" perform MTU discovery to the RP, and keep those discoveries inside 631.\" the kernel so the encapsulating router may send back ICMP 632.\" Fragmentation Required if the size of the multicast data packet is 633.\" too large (see "Encapsulating data packets in the Register Tunnel" 634.\" in Section 4.4.1 in the PIM-SM spec 635.\" draft-ietf-pim-sm-v2-new-05.{txt,ps}). 636.\" For IPv4 we may be able to get away without it, but for IPv6 we need 637.\" that. 638.\" 639.\" 4. Mechanism for "multicast bandwidth monitoring and upcalls". 640.\" 641.Pp 642Typically, a multicast routing user-level process would need to know the 643forwarding bandwidth for some data flow. 644For example, the multicast routing process may want to timeout idle MFC 645entries, or in case of PIM-SM it can initiate (S,G) shortest-path switch if 646the bandwidth rate is above a threshold for example. 647.Pp 648The original solution for measuring the bandwidth of a dataflow was 649that a user-level process would periodically 650query the kernel about the number of forwarded packets/bytes per 651(S,G), and then based on those numbers it would estimate whether a source 652has been idle, or whether the source's transmission bandwidth is above a 653threshold. 654That solution is far from being scalable, hence the need for a new 655mechanism for bandwidth monitoring. 656.Pp 657Below is a description of the bandwidth monitoring mechanism. 658.Bl -bullet 659.It 660If the bandwidth of a data flow satisfies some pre-defined filter, 661the kernel delivers an upcall on the multicast routing socket 662to the multicast routing process that has installed that filter. 663.It 664The bandwidth-upcall filters are installed per (S,G). There can be 665more than one filter per (S,G). 666.It 667Instead of supporting all possible comparison operations 668(i.e., < <= == != > >= ), there is support only for the 669<= and >= operations, 670because this makes the kernel-level implementation simpler, 671and because practically we need only those two. 672Further, the missing operations can be simulated by secondary 673user-level filtering of those <= and >= filters. 674For example, to simulate !=, then we need to install filter 675.Dq bw <= 0xffffffff , 676and after an 677upcall is received, we need to check whether 678.Dq measured_bw != expected_bw . 679.It 680The bandwidth-upcall mechanism is enabled by 681setsockopt(MRT_API_CONFIG) for the MRT_MFC_BW_UPCALL flag. 682.It 683The bandwidth-upcall filters are added/deleted by the new 684setsockopt(MRT_ADD_BW_UPCALL) and setsockopt(MRT_DEL_BW_UPCALL) 685respectively (with the appropriate 686.Dq struct bw_upcall 687argument of course). 688.El 689.Pp 690From application point of view, a developer needs to know about 691the following: 692.Bd -literal 693/* 694 * Structure for installing or delivering an upcall if the 695 * measured bandwidth is above or below a threshold. 696 * 697 * User programs (e.g. daemons) may have a need to know when the 698 * bandwidth used by some data flow is above or below some threshold. 699 * This interface allows the userland to specify the threshold (in 700 * bytes and/or packets) and the measurement interval. Flows are 701 * all packet with the same source and destination IP address. 702 * At the moment the code is only used for multicast destinations 703 * but there is nothing that prevents its use for unicast. 704 * 705 * The measurement interval cannot be shorter than some Tmin (currently, 3s). 706 * The threshold is set in packets and/or bytes per_interval. 707 * 708 * Measurement works as follows: 709 * 710 * For >= measurements: 711 * The first packet marks the start of a measurement interval. 712 * During an interval we count packets and bytes, and when we 713 * pass the threshold we deliver an upcall and we are done. 714 * The first packet after the end of the interval resets the 715 * count and restarts the measurement. 716 * 717 * For <= measurement: 718 * We start a timer to fire at the end of the interval, and 719 * then for each incoming packet we count packets and bytes. 720 * When the timer fires, we compare the value with the threshold, 721 * schedule an upcall if we are below, and restart the measurement 722 * (reschedule timer and zero counters). 723 */ 724 725struct bw_data { 726 struct timeval b_time; 727 uint64_t b_packets; 728 uint64_t b_bytes; 729}; 730 731struct bw_upcall { 732 struct in_addr bu_src; /* source address */ 733 struct in_addr bu_dst; /* destination address */ 734 uint32_t bu_flags; /* misc flags (see below) */ 735#define BW_UPCALL_UNIT_PACKETS (1 << 0) /* threshold (in packets) */ 736#define BW_UPCALL_UNIT_BYTES (1 << 1) /* threshold (in bytes) */ 737#define BW_UPCALL_GEQ (1 << 2) /* upcall if bw >= threshold */ 738#define BW_UPCALL_LEQ (1 << 3) /* upcall if bw <= threshold */ 739#define BW_UPCALL_DELETE_ALL (1 << 4) /* delete all upcalls for s,d*/ 740 struct bw_data bu_threshold; /* the bw threshold */ 741 struct bw_data bu_measured; /* the measured bw */ 742}; 743 744/* max. number of upcalls to deliver together */ 745#define BW_UPCALLS_MAX 128 746/* min. threshold time interval for bandwidth measurement */ 747#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_SEC 3 748#define BW_UPCALL_THRESHOLD_INTERVAL_MIN_USEC 0 749.Ed 750.Pp 751The 752.Dq bw_upcall 753structure is used as an argument to 754setsockopt(MRT_ADD_BW_UPCALL) and setsockopt(MRT_DEL_BW_UPCALL). 755Each setsockopt(MRT_ADD_BW_UPCALL) installs a filter in the kernel 756for the source and destination address in the 757.Dq bw_upcall 758argument, 759and that filter will trigger an upcall according to the following 760pseudo-algorithm: 761.Bd -literal 762 if (bw_upcall_oper IS ">=") { 763 if (((bw_upcall_unit & PACKETS == PACKETS) && 764 (measured_packets >= threshold_packets)) || 765 ((bw_upcall_unit & BYTES == BYTES) && 766 (measured_bytes >= threshold_bytes))) 767 SEND_UPCALL("measured bandwidth is >= threshold"); 768 } 769 if (bw_upcall_oper IS "<=" && measured_interval >= threshold_interval) { 770 if (((bw_upcall_unit & PACKETS == PACKETS) && 771 (measured_packets <= threshold_packets)) || 772 ((bw_upcall_unit & BYTES == BYTES) && 773 (measured_bytes <= threshold_bytes))) 774 SEND_UPCALL("measured bandwidth is <= threshold"); 775 } 776.Ed 777.Pp 778In the same 779.Dq bw_upcall 780the unit can be specified in both BYTES and PACKETS. 781However, the GEQ and LEQ flags are mutually exclusive. 782.Pp 783Basically, an upcall is delivered if the measured bandwidth is >= or 784<= the threshold bandwidth (within the specified measurement 785interval). 786For practical reasons, the smallest value for the measurement 787interval is 3 seconds. 788If smaller values are allowed, then the bandwidth 789estimation may be less accurate, or the potentially very high frequency 790of the generated upcalls may introduce too much overhead. 791For the >= operation, the answer may be known before the end of 792.Dq threshold_interval , 793therefore the upcall may be delivered earlier. 794For the <= operation however, we must wait 795until the threshold interval has expired to know the answer. 796.Pp 797Example of usage: 798.Bd -literal 799struct bw_upcall bw_upcall; 800/* Assign all bw_upcall fields as appropriate */ 801memset(&bw_upcall, 0, sizeof(bw_upcall)); 802memcpy(&bw_upcall.bu_src, &source, sizeof(bw_upcall.bu_src)); 803memcpy(&bw_upcall.bu_dst, &group, sizeof(bw_upcall.bu_dst)); 804bw_upcall.bu_threshold.b_data = threshold_interval; 805bw_upcall.bu_threshold.b_packets = threshold_packets; 806bw_upcall.bu_threshold.b_bytes = threshold_bytes; 807if (is_threshold_in_packets) 808 bw_upcall.bu_flags |= BW_UPCALL_UNIT_PACKETS; 809if (is_threshold_in_bytes) 810 bw_upcall.bu_flags |= BW_UPCALL_UNIT_BYTES; 811do { 812 if (is_geq_upcall) { 813 bw_upcall.bu_flags |= BW_UPCALL_GEQ; 814 break; 815 } 816 if (is_leq_upcall) { 817 bw_upcall.bu_flags |= BW_UPCALL_LEQ; 818 break; 819 } 820 return (ERROR); 821} while (0); 822setsockopt(mrouter_s4, IPPROTO_IP, MRT_ADD_BW_UPCALL, 823 (void *)&bw_upcall, sizeof(bw_upcall)); 824.Ed 825.Pp 826To delete a single filter, then use MRT_DEL_BW_UPCALL, 827and the fields of bw_upcall must be set 828exactly same as when MRT_ADD_BW_UPCALL was called. 829.Pp 830To delete all bandwidth filters for a given (S,G), then 831only the 832.Dq bu_src 833and 834.Dq bu_dst 835fields in 836.Dq struct bw_upcall 837need to be set, and then just set only the 838.Dq BW_UPCALL_DELETE_ALL 839flag inside field 840.Dq bw_upcall.bu_flags . 841.Pp 842The bandwidth upcalls are received by aggregating them in the new upcall 843message: 844.Bd -literal 845#define IGMPMSG_BW_UPCALL 4 /* BW monitoring upcall */ 846.Ed 847.Pp 848This message is an array of 849.Dq struct bw_upcall 850elements (up to BW_UPCALLS_MAX = 128). 851The upcalls are 852delivered when there are 128 pending upcalls, or when 1 second has 853expired since the previous upcall (whichever comes first). 854In an 855.Dq struct upcall 856element, the 857.Dq bu_measured 858field is filled-in to 859indicate the particular measured values. 860However, because of the way 861the particular intervals are measured, the user should be careful how 862bu_measured.b_time is used. 863For example, if the 864filter is installed to trigger an upcall if the number of packets 865is >= 1, then 866.Dq bu_measured 867may have a value of zero in the upcalls after the 868first one, because the measured interval for >= filters is 869.Dq clocked 870by the forwarded packets. 871Hence, this upcall mechanism should not be used for measuring 872the exact value of the bandwidth of the forwarded data. 873To measure the exact bandwidth, the user would need to 874get the forwarded packets statistics with the ioctl(SIOCGETSGCNT) 875mechanism 876(see the 877.Sx Programming Guide 878section) . 879.Pp 880Note that the upcalls for a filter are delivered until the specific 881filter is deleted, but no more frequently than once per 882.Dq bu_threshold.b_time . 883For example, if the filter is specified to 884deliver a signal if bw >= 1 packet, the first packet will trigger a 885signal, but the next upcall will be triggered no earlier than 886.Dq bu_threshold.b_time 887after the previous upcall. 888.\" 889.Sh SEE ALSO 890.Xr getsockopt 2 , 891.Xr recvfrom 2 , 892.Xr recvmsg 2 , 893.Xr setsockopt 2 , 894.Xr socket 2 , 895.Xr icmp6 4 , 896.Xr inet 4 , 897.Xr inet6 4 , 898.Xr intro 4 , 899.Xr ip 4 , 900.Xr ip6 4 , 901.Xr pim 4 902.\" 903.Sh AUTHORS 904.An -nosplit 905The original multicast code was written by 906.An David Waitzman 907(BBN Labs), and later modified by the following individuals: 908.An Steve Deering 909(Stanford), 910.An Mark J. Steiglitz 911(Stanford), 912.An Van Jacobson 913(LBL), 914.An Ajit Thyagarajan 915(PARC), 916.An Bill Fenner 917(PARC). 918The IPv6 multicast support was implemented by the KAME project 919.Pa ( http://www.kame.net ) , 920and was based on the IPv4 multicast code. 921The advanced multicast API and the multicast bandwidth 922monitoring were implemented by 923.An Pavlin Radoslavov 924(ICSI) in collaboration with 925.An Chris Brown 926(NextHop). 927.Pp 928This manual page was written by 929.An Pavlin Radoslavov 930(ICSI). 931