1.\" $NetBSD: altq.9,v 1.8 2002/05/28 11:41:45 wiz Exp $ 2.\" $OpenBSD: altq.9,v 1.4 2001/07/12 12:41:42 itojun Exp $ 3.\" 4.\" Copyright (C) 2004 Max Laier. All rights reserved. 5.\" Copyright (C) 2001 6.\" Sony Computer Science Laboratories Inc. All rights reserved. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in the 15.\" documentation and/or other materials provided with the distribution. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY SONY CSL AND CONTRIBUTORS ``AS IS'' AND 18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL SONY CSL OR CONTRIBUTORS BE LIABLE 21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 27.\" SUCH DAMAGE. 28.\" 29.\" $FreeBSD$ 30.\" 31.Dd August 25, 2004 32.Dt ALTQ 9 33.Os 34.\" 35.Sh NAME 36.Nm ALTQ 37.Nd kernel interfaces for manipulating output queues on network interfaces 38.Sh SYNOPSIS 39.In sys/types.h 40.In sys/socket.h 41.In net/if.h 42.In net/if_var.h 43.\" 44.Ss Enqueue macros 45.Fn IFQ_ENQUEUE "struct ifaltq *ifq" "struct mbuf *m" "int error" 46.Fn IFQ_HANDOFF "struct ifnet *ifp" "struct mbuf *m" "int error" 47.Fo IFQ_HANDOFF_ADJ 48.Fa "struct ifnet *ifp" "struct mbuf *m" "int adjust" "int error" 49.Fc 50.\" 51.Ss Dequeue macros 52.Fn IFQ_DEQUEUE "struct ifaltq *ifq" "struct mbuf *m" 53.Fn IFQ_POLL_NOLOCK "struct ifaltq *ifq" "struct mbuf *m" 54.Fn IFQ_PURGE "struct ifaltq *ifq" 55.Fn IFQ_IS_EMPTY "struct ifaltq *ifq" 56.\" 57.Ss Driver managed dequeue macros 58.Fn IFQ_DRV_DEQUEUE "struct ifaltq *ifq" "struct mbuf *m" 59.Fn IFQ_DRV_PREPEND "struct ifaltq *ifq" "struct mbuf *m" 60.Fn IFQ_DRV_PURGE "struct ifaltq *ifq" 61.Fn IFQ_DRV_IS_EMPTY "struct ifaltq *ifq" 62.\" 63.Ss General setup macros 64.Fn IFQ_SET_MAXLEN "struct ifaltq *ifq" "int len" 65.Fn IFQ_INC_LEN "struct ifaltq *ifq" 66.Fn IFQ_DEC_LEN "struct ifaltq *ifq" 67.Fn IFQ_INC_DROPS "struct ifaltq *ifq" 68.Fn IFQ_SET_READY "struct ifaltq *ifq" 69.Sh DESCRIPTION 70The 71.Nm 72system is a framework to manage queuing disciplines on network 73interfaces. 74.Nm 75introduces new macros to manipulate output queues. 76The output queue macros are used to abstract queue operations and not to 77touch the internal fields of the output queue structure. 78The macros are independent from the 79.Nm 80implementation, and compatible with the traditional 81.Vt ifqueue 82macros for ease of transition. 83.Pp 84.Fn IFQ_ENQUEUE , 85.Fn IFQ_HANDOFF 86and 87.Fn IFQ_HANDOFF_ADJ 88enqueue a packet 89.Fa m 90to the queue 91.Fa ifq . 92The underlying queuing discipline may discard the packet. 93The 94.Fa error 95argument is set to 0 on success, or 96.Er ENOBUFS 97if the packet is discarded. 98The packet pointed to by 99.Fa m 100will be freed by the device driver on success, or by the queuing discipline on 101failure, so the caller should not touch 102.Fa m 103after enqueuing. 104.Fn IFQ_HANDOFF 105and 106.Fn IFQ_HANDOFF_ADJ 107combine the enqueue operation with statistic generation and call 108.Fn if_start 109upon successful enqueue to initiate the actual send. 110.Pp 111.Fn IFQ_DEQUEUE 112dequeues a packet from the queue. 113The dequeued packet is returned in 114.Fa m , 115or 116.Fa m 117is set to 118.Dv NULL 119if no packet is dequeued. 120The caller must always check 121.Fa m 122since a non-empty queue could return 123.Dv NULL 124under rate-limiting. 125.Pp 126.Fn IFQ_POLL_NOLOCK 127returns the next packet without removing it from the queue. 128The caller must hold the queue mutex when calling 129.Fn IFQ_POLL_NOLOCK 130in order to guarantee that a subsequent call to 131.Fn IFQ_DEQUEUE_NOLOCK 132dequeues the same packet. 133.Pp 134.Fn IFQ_*_NOLOCK 135variants (if available) always assume that the caller holds the queue mutex. 136They can be grabbed with 137.Fn IFQ_LOCK 138and released with 139.Fn IFQ_UNLOCK . 140.Pp 141.Fn IFQ_PURGE 142discards all the packets in the queue. 143The purge operation is needed since a non-work conserving queue cannot be 144emptied by a dequeue loop. 145.Pp 146.Fn IFQ_IS_EMPTY 147can be used to check if the queue is empty. 148Note that 149.Fn IFQ_DEQUEUE 150could still return 151.Dv NULL 152if the queuing discipline is non-work conserving. 153.Pp 154.Fn IFQ_DRV_DEQUEUE 155moves up to 156.Fa ifq->ifq_drv_maxlen 157packets from the queue to the 158.Dq "driver managed" 159queue and returns the first one via 160.Fa m . 161As for 162.Fn IFQ_DEQUEUE , 163.Fa m 164can be 165.Dv NULL 166even for a non-empty queue. 167Subsequent calls to 168.Fn IFQ_DRV_DEQUEUE 169pass the packets from the 170.Dq "driver managed" 171queue without obtaining the queue mutex. 172It is the responsibility of the caller to protect against concurrent access. 173Enabling 174.Nm 175for a given queue sets 176.Va ifq_drv_maxlen 177to 0 as the 178.Dq "bulk dequeue" 179performed by 180.Fn IFQ_DRV_DEQUEUE 181for higher values of 182.Va ifq_drv_maxlen 183is adverse to 184.Nm ALTQ Ns 's 185internal timing. 186Note that a driver must not mix 187.Fn IFQ_DRV_* 188macros with the default dequeue macros as the default macros do not look at the 189.Dq "driver managed" 190queue which might lead to an mbuf leak. 191.Pp 192.Fn IFQ_DRV_PREPEND 193prepends 194.Fa m 195to the 196.Dq "driver managed" 197queue from where it will be obtained with the next call to 198.Fn IFQ_DRV_DEQUEUE . 199.Pp 200.Fn IFQ_DRV_PURGE 201flushes all packets in the 202.Dq "driver managed" 203queue and calls to 204.Fn IFQ_PURGE 205afterwards. 206.Pp 207.Fn IFQ_DRV_IS_EMPTY 208checks for packets in the 209.Dq "driver managed" 210part of the queue. 211If it is empty, it forwards to 212.Fn IFQ_IS_EMPTY . 213.Pp 214.Fn IFQ_SET_MAXLEN 215sets the queue length limit to the default FIFO queue. 216The 217.Va ifq_drv_maxlen 218member of the 219.Vt ifaltq 220structure controls the length limit of the 221.Dq "driver managed" 222queue. 223.Pp 224.Fn IFQ_INC_LEN 225and 226.Fn IFQ_DEC_LEN 227increment or decrement the current queue length in packets. 228This is mostly for internal purposes. 229.Pp 230.Fn IFQ_INC_DROPS 231increments the drop counter and is identical to 232.Fn IF_DROP . 233It is defined for naming consistency only. 234.Pp 235.Fn IFQ_SET_READY 236sets a flag to indicate that a driver was converted to use the new macros. 237.Nm 238can be enabled only on interfaces with this flag. 239.Sh COMPATIBILITY 240.Ss Vt ifaltq Ss structure 241In order to keep compatibility with the existing code, the new 242output queue structure 243.Vt ifaltq 244has the same fields. 245The traditional 246.Fn IF_* 247macros and the code directly referencing the fields within 248.Va if_snd 249still work with 250.Vt ifaltq . 251.Bd -literal 252 ##old-style## ##new-style## 253 | 254 struct ifqueue { | struct ifaltq { 255 struct mbuf *ifq_head; | struct mbuf *ifq_head; 256 struct mbuf *ifq_tail; | struct mbuf *ifq_tail; 257 int ifq_len; | int ifq_len; 258 int ifq_maxlen; | int ifq_maxlen; 259 int ifq_drops; | int ifq_drops; 260 }; | /* driver queue fields */ 261 | ...... 262 | /* altq related fields */ 263 | ...... 264 | }; 265 | 266.Ed 267The new structure replaces 268.Vt "struct ifqueue" 269in 270.Vt "struct ifnet" . 271.Bd -literal 272 ##old-style## ##new-style## 273 | 274 struct ifnet { | struct ifnet { 275 .... | .... 276 | 277 struct ifqueue if_snd; | struct ifaltq if_snd; 278 | 279 .... | .... 280 }; | }; 281 | 282.Ed 283The (simplified) new 284.Fn IFQ_* 285macros look like: 286.Bd -literal 287 #define IFQ_DEQUEUE(ifq, m) \e 288 if (ALTQ_IS_ENABLED((ifq)) \e 289 ALTQ_DEQUEUE((ifq), (m)); \e 290 else \e 291 IF_DEQUEUE((ifq), (m)); 292.Ed 293.Ss Enqueue operation 294The semantics of the enqueue operation is changed. 295In the new style, 296enqueue and packet drop are combined since they cannot be easily 297separated in many queuing disciplines. 298The new enqueue operation corresponds to the following macro that is 299written with the old macros. 300.Bd -literal 301#define IFQ_ENQUEUE(ifq, m, error) \e 302do { \e 303 if (IF_QFULL((ifq))) { \e 304 m_freem((m)); \e 305 (error) = ENOBUFS; \e 306 IF_DROP(ifq); \e 307 } else { \e 308 IF_ENQUEUE((ifq), (m)); \e 309 (error) = 0; \e 310 } \e 311} while (0) 312.Ed 313.Pp 314.Fn IFQ_ENQUEUE 315does the following: 316.Pp 317.Bl -hyphen -compact 318.It 319queue a packet, 320.It 321drop (and free) a packet if the enqueue operation fails. 322.El 323.Pp 324If the enqueue operation fails, 325.Fa error 326is set to 327.Er ENOBUFS . 328The 329.Fa m 330mbuf is freed by the queuing discipline. 331The caller should not touch mbuf after calling 332.Fn IFQ_ENQUEUE 333so that the caller may need to copy 334.Va m_pkthdr.len 335or 336.Va m_flags 337field beforehand for statistics. 338.Fn IFQ_HANDOFF 339and 340.Fn IFQ_HANDOFF_ADJ 341can be used if only default interface statistics and an immediate call to 342.Fn if_start 343are desired. 344The caller should not use 345.Fn senderr 346since mbuf was already freed. 347.Pp 348The new style 349.Fn if_output 350looks as follows: 351.Bd -literal 352 ##old-style## ##new-style## 353 | 354 int | int 355 ether_output(ifp, m0, dst, rt0) | ether_output(ifp, m0, dst, rt0) 356 { | { 357 ...... | ...... 358 | 359 | mflags = m->m_flags; 360 | len = m->m_pkthdr.len; 361 s = splimp(); | s = splimp(); 362 if (IF_QFULL(&ifp->if_snd)) { | IFQ_ENQUEUE(&ifp->if_snd, m, 363 | error); 364 IF_DROP(&ifp->if_snd); | if (error != 0) { 365 splx(s); | splx(s); 366 senderr(ENOBUFS); | return (error); 367 } | } 368 IF_ENQUEUE(&ifp->if_snd, m); | 369 ifp->if_obytes += | ifp->if_obytes += len; 370 m->m_pkthdr.len; | 371 if (m->m_flags & M_MCAST) | if (mflags & M_MCAST) 372 ifp->if_omcasts++; | ifp->if_omcasts++; 373 | 374 if ((ifp->if_flags & IFF_OACTIVE) | if ((ifp->if_flags & IFF_OACTIVE) 375 == 0) | == 0) 376 (*ifp->if_start)(ifp); | (*ifp->if_start)(ifp); 377 splx(s); | splx(s); 378 return (error); | return (error); 379 | 380 bad: | bad: 381 if (m) | if (m) 382 m_freem(m); | m_freem(m); 383 return (error); | return (error); 384 } | } 385 | 386.Ed 387.Sh HOW TO CONVERT THE EXISTING DRIVERS 388First, make sure the corresponding 389.Fn if_output 390is already converted to the new style. 391.Pp 392Look for 393.Va if_snd 394in the driver. 395Probably, you need to make changes to the lines that include 396.Va if_snd . 397.Ss Empty check operation 398If the code checks 399.Va ifq_head 400to see whether the queue is empty or not, use 401.Fn IFQ_IS_EMPTY . 402.Bd -literal 403 ##old-style## ##new-style## 404 | 405 if (ifp->if_snd.ifq_head != NULL) | if (!IFQ_IS_EMPTY(&ifp->if_snd)) 406 | 407.Ed 408.Fn IFQ_IS_EMPTY 409only checks if there is any packet stored in the queue. 410Note that even when 411.Fn IFQ_IS_EMPTY 412is 413.Dv FALSE , 414.Fn IFQ_DEQUEUE 415could still return 416.Dv NULL 417if the queue is under rate-limiting. 418.Ss Dequeue operation 419Replace 420.Fn IF_DEQUEUE 421by 422.Fn IFQ_DEQUEUE . 423Always check whether the dequeued mbuf is 424.Dv NULL 425or not. 426Note that even when 427.Fn IFQ_IS_EMPTY 428is 429.Dv FALSE , 430.Fn IFQ_DEQUEUE 431could return 432.Dv NULL 433due to rate-limiting. 434.Bd -literal 435 ##old-style## ##new-style## 436 | 437 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_DEQUEUE(&ifp->if_snd, m); 438 | if (m == NULL) 439 | return; 440 | 441.Ed 442A driver is supposed to call 443.Fn if_start 444from transmission complete interrupts in order to trigger the next dequeue. 445.Ss Poll-and-dequeue operation 446If the code polls the packet at the head of the queue and actually uses 447the packet before dequeuing it, use 448.Fn IFQ_POLL_NOLOCK 449and 450.Fn IFQ_DEQUEUE_NOLOCK . 451.Bd -literal 452 ##old-style## ##new-style## 453 | 454 | IFQ_LOCK(&ifp->if_snd); 455 m = ifp->if_snd.ifq_head; | IFQ_POLL_NOLOCK(&ifp->if_snd, m); 456 if (m != NULL) { | if (m != NULL) { 457 | 458 /* use m to get resources */ | /* use m to get resources */ 459 if (something goes wrong) | if (something goes wrong) 460 | IFQ_UNLOCK(&ifp->if_snd); 461 return; | return; 462 | 463 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_DEQUEUE_NOLOCK(&ifp->if_snd, m); 464 | IFQ_UNLOCK(&ifp->if_snd); 465 | 466 /* kick the hardware */ | /* kick the hardware */ 467 } | } 468 | 469.Ed 470It is guaranteed that 471.Fn IFQ_DEQUEUE_NOLOCK 472under the same lock as a previous 473.Fn IFQ_POLL_NOLOCK 474returns the same packet. 475Note that they need to be guarded by 476.Fn IFQ_LOCK . 477.Ss Eliminating Fn IF_PREPEND 478If the code uses 479.Fn IF_PREPEND , 480you have to eliminate it unless you can use a 481.Dq "driver managed" 482queue which allows the use of 483.Fn IFQ_DRV_PREPEND 484as a substitute. 485A common usage of 486.Fn IF_PREPEND 487is to cancel the previous dequeue operation. 488You have to convert the logic into poll-and-dequeue. 489.Bd -literal 490 ##old-style## ##new-style## 491 | 492 | IFQ_LOCK(&ifp->if_snd); 493 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_POLL_NOLOCK(&ifp->if_snd, m); 494 if (m != NULL) { | if (m != NULL) { 495 | 496 if (something_goes_wrong) { | if (something_goes_wrong) { 497 IF_PREPEND(&ifp->if_snd, m); | IFQ_UNLOCK(&ifp->if_snd); 498 return; | return; 499 } | } 500 | 501 | /* at this point, the driver 502 | * is committed to send this 503 | * packet. 504 | */ 505 | IFQ_DEQUEUE_NOLOCK(&ifp->if_snd, m); 506 | IFQ_UNLOCK(&ifp->if_snd); 507 | 508 /* kick the hardware */ | /* kick the hardware */ 509 } | } 510 | 511.Ed 512.Ss Purge operation 513Use 514.Fn IFQ_PURGE 515to empty the queue. 516Note that a non-work conserving queue cannot be emptied by a dequeue loop. 517.Bd -literal 518 ##old-style## ##new-style## 519 | 520 while (ifp->if_snd.ifq_head != NULL) {| IFQ_PURGE(&ifp->if_snd); 521 IF_DEQUEUE(&ifp->if_snd, m); | 522 m_freem(m); | 523 } | 524 | 525.Ed 526.Ss Conversion using a driver managed queue 527Convert 528.Fn IF_* 529macros to their equivalent 530.Fn IFQ_DRV_* 531and employ 532.Fn IFQ_DRV_IS_EMPTY 533where appropriate. 534.Bd -literal 535 ##old-style## ##new-style## 536 | 537 if (ifp->if_snd.ifq_head != NULL) | if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) 538 | 539.Ed 540Make sure that calls to 541.Fn IFQ_DRV_DEQUEUE , 542.Fn IFQ_DRV_PREPEND 543and 544.Fn IFQ_DRV_PURGE 545are protected with a mutex of some kind. 546.Ss Attach routine 547Use 548.Fn IFQ_SET_MAXLEN 549to set 550.Va ifq_maxlen 551to 552.Fa len . 553Initialize 554.Va ifq_drv_maxlen 555with a sensible value if you plan to use the 556.Fn IFQ_DRV_* 557macros. 558Add 559.Fn IFQ_SET_READY 560to show this driver is converted to the new style. 561(This is used to distinguish new-style drivers.) 562.Bd -literal 563 ##old-style## ##new-style## 564 | 565 ifp->if_snd.ifq_maxlen = qsize; | IFQ_SET_MAXLEN(&ifp->if_snd, qsize); 566 | ifp->if_snd.ifq_drv_maxlen = qsize; 567 | IFQ_SET_READY(&ifp->if_snd); 568 if_attach(ifp); | if_attach(ifp); 569 | 570.Ed 571.Ss Other issues 572The new macros for statistics: 573.Bd -literal 574 ##old-style## ##new-style## 575 | 576 IF_DROP(&ifp->if_snd); | IFQ_INC_DROPS(&ifp->if_snd); 577 | 578 ifp->if_snd.ifq_len++; | IFQ_INC_LEN(&ifp->if_snd); 579 | 580 ifp->if_snd.ifq_len--; | IFQ_DEC_LEN(&ifp->if_snd); 581 | 582.Ed 583.Sh QUEUING DISCIPLINES 584Queuing disciplines need to maintain 585.Fa ifq_len 586(used by 587.Fn IFQ_IS_EMPTY ) . 588Queuing disciplines also need to guarantee that the same mbuf is returned if 589.Fn IFQ_DEQUEUE 590is called immediately after 591.Fn IFQ_POLL . 592.Sh SEE ALSO 593.Xr pf 4 , 594.Xr pf.conf 5 , 595.Xr pfctl 8 596.Sh HISTORY 597The 598.Nm 599system first appeared in March 1997. 600