1.\" 2.\" Copyright (c) 2020, Gordon Bergling <gbe@FreeBSD.org> 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR 17.\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.Dd November 7, 2022 26.Dt TCP_BBR 4 27.Os 28.Sh NAME 29.Nm tcp_bbr 30.Nd TCP Bottleneck Bandwidth and Round-Trip Time Algorithm 31.Sh SYNOPSIS 32To use this TCP stack you have to place the following line in your 33kernel configuration file: 34.Bd -ragged -offset indent 35.Cd "options TCPHPTS" 36.Ed 37.Pp 38To load the driver as a module at boot time, place the following line in 39.Xr loader.conf 5 : 40.Bd -literal -offset indent 41tcp_bbr_load="YES" 42.Ed 43.Pp 44To enable the TCP stack you must place the following line in the 45.Xr sysctl.conf 5 : 46.Bd -literal -offset indent 47net.inet.tcp.functions_default=bbr 48.Ed 49.Sh DESCRIPTION 50Bottleneck bandwidth and round-trip time (BBR) is a congestion control algorithm which 51seeks high throughput with a small queue by probing BW and RTT. 52It is a round-up redesign of congestion control, which is not loss-based, delay-based, 53ECN-based or AIMD-based. 54.Pp 55The core design of BBR is about creating a model graph of the network path by 56estimating the maximum BW and minimum RTT on each ACK. 57.Sh MIB Variables 58The algorithm exposes the following scopes in the 59.Va net.inet.tcp.bbr 60branch of the 61.Xr sysctl 3 62MIB: 63.Bl -tag -width ".Va exp_backoff_scale" 64.It Va cwnd 65Cwnd controls, for example "target cwnd rtt measurement" and "BBR initial window". 66.It Va measure 67Measurement controls. 68.It Va pacing 69Connection pacing controls. 70.It Va policer 71Policer controls, for example "false detection threshold" and "loss threshold". 72.It Va probertt 73Probe RTT controls. 74.It Va startup 75Startup controls. 76.It Va states 77State controls. 78.It Va timeout 79Time out controls. 80.El 81.Pp 82Besides the variables within the above scopes the following 83variables are also exposed in the 84.Va net.inet.tcp.bbr 85branch: 86.Bl -tag -width ".Va exp_backoff_scale" 87.It Va clrlost 88Clear lost counters. 89.It Va software_pacing 90Total number of software paced flows. 91.It Va hdwr_pacing 92Total number of hardware paced flows. 93.It Va enob_no_hdwr_pacing 94Total number of enobufs for non-hardware paced flows. 95.It Va enob_hdwr_pacing 96Total number of enobufs for hardware paced flows. 97.It Va rtt_tlp_thresh 98What divisor for TLP rtt/retran will be added (1=rtt, 2=1/2 rtt etc). 99.It Va reorder_fade 100Does reorder detection fade, if so how many ms (0 means never). 101.It Va reorder_thresh 102What factor for rack will be added when seeing reordering (shift right). 103.It Va bb_verbose 104Should BBR black box logging be verbose. 105.It Va sblklimit 106When do we start ignoring small sack blocks. 107.It Va resend_use_tso 108Can resends use TSO? 109.It Va data_after_close 110Do we hold off sending a RST until all pending data is ack'd. 111.It Va kill_paceout 112When we hit this many errors in a row, kill the session? 113.It Va error_paceout 114When we hit an error what is the min to pace out in usec's? 115.It Va cheat_rxt 116Do we burst 1ms between sends on retransmissions (like rack)? 117.It Va minrto 118Minimum RTO in ms. 119.El 120.Sh SEE ALSO 121.Xr cc_chd 4 , 122.Xr cc_cubic 4 , 123.Xr cc_hd 4 , 124.Xr cc_htcp 4 , 125.Xr cc_newreno 4 , 126.Xr cc_vegas 4 , 127.Xr h_ertt 4 , 128.Xr mod_cc 4 , 129.Xr tcp 4 , 130.Xr tcp_rack 4 , 131.Xr mod_cc 9 132.Rs 133.%A "Neal Cardwell" 134.%A "Yuchung Cheng" 135.%A "Stephen Gunn" 136.%A "Soheil Hassas Yeganeh" 137.%A "Van Jacobson" 138.%T "BBR: Congestion-Based Congestion Control" 139.%J "ACM Queue, Vol. 14" 140.%D "September / October 2016" 141.Re 142.Rs 143.%A "Dominik Scholz" 144.%A "Benedikt Jaeger" 145.%A "Lukas Schwaighofer" 146.%A "Daniel Raumer" 147.%A "Fabien Geyer" 148.%A "Georg Carle" 149.%T "Towards a Deeper Understanding of TCP BBR Congestion Control" 150.%J "IFIP Networking 2018" 151.%D "May 2018" 152.%U "http://www.net.in.tum.de/fileadmin/bibtex/publications/papers/IFIP-Networking-2018-TCP-BBR.pdf" 153.Re 154.Sh HISTORY 155The 156.Nm 157congestion control module first appeared in 158.Fx 13.0 . 159.Sh AUTHORS 160.An -nosplit 161The 162.Nm 163congestion control module was written by 164.An Randall Stewart Aq Mt rrs@FreeBSD.org 165and sponsored by Netflix, Inc. 166This manual page was written by 167.An Gordon Bergling Aq Mt gbe@FreeBSD.org . 168