1.\" 2.\" Copyright (c) 2003, Ryan McBride. All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $ 26.\" $FreeBSD: src/share/man/man4/carp.4,v 1.10 2006/06/07 10:26:51 glebius Exp $ 27.\" $DragonFly: src/share/man/man4/carp.4,v 1.4 2008/10/17 12:41:38 swildner Exp $ 28.\" 29.Dd April 9, 2007 30.Dt CARP 4 31.Os 32.Sh NAME 33.Nm carp 34.Nd Common Address Redundancy Protocol 35.Sh SYNOPSIS 36.Cd "options CARP" 37.Cd "pseudo-device carp" 38.Sh DESCRIPTION 39The 40.Nm 41interface is a pseudo-device that implements and controls the 42CARP protocol. 43CARP allows multiple hosts on the same local network to share a set of IP addresses. 44Its primary purpose is to ensure that these 45addresses are always available, but in some configurations 46.Nm 47can also provide load balancing functionality. 48.Pp 49A 50.Nm 51interface can be created at runtime using the 52.Nm ifconfig Li carp Ns Ar N Cm create 53command or by configuring 54it via 55.Va cloned_interfaces 56in the 57.Pa /etc/rc.conf 58file. 59.Pp 60To use 61.Nm , 62the administrator needs to configure at minimum a common virtual host ID (VHID) 63and virtual host IP address on each machine which is to take part in the virtual 64group. 65Additional parameters can also be set on a per-interface basis: 66.Cm advbase 67and 68.Cm advskew , 69which are used to control how frequently the host sends advertisements when it 70is the master for a virtual host, and 71.Cm pass 72which is used to authenticate 73.Nm 74advertisements. 75The 76.Cm advbase 77parameter stands for 78.Dq "advertisement base" . 79It is measured in seconds and specifies the base of the advertisement interval. 80The 81.Cm advskew 82parameter stands for 83.Dq "advertisement skew" . 84It is measured in 1/256 of seconds. 85It is added to the base advertisement interval to make one host advertise 86a bit slower than the other does. 87Both 88.Cm advbase 89and 90.Cm advskew 91are put inside CARP advertisements. 92These configurations can be done using 93.Xr ifconfig 8 , 94or through the 95.Dv SIOCSVH 96.Xr ioctl 2 . 97.Pp 98Additionally, there are a number of global parameters which can be set using 99.Xr sysctl 8 : 100.Bl -tag -width ".Va net.inet.carp.arpbalance" 101.It Va net.inet.carp.allow 102Accept incoming 103.Nm 104packets. 105Enabled by default. 106.It Va net.inet.carp.preempt 107Allow virtual hosts to preempt each other. 108It is also used to failover 109.Nm 110interfaces as a group. 111When the option is enabled and one of the 112.Nm 113enabled physical interfaces 114goes down, 115.Cm advskew 116is changed to 240 on all 117.Nm 118interfaces. 119See also the first example. 120Disabled by default. 121.It Va net.inet.carp.log 122Value of 0 disables any logging. 123Value of 1 enables logging of bad 124.Nm 125packets. 126Values above 1 enable logging state changes of 127.Nm 128interfaces. 129Default value is 1. 130.It Va net.inet.carp.arpbalance 131Balance local traffic using ARP (see below). 132Disabled by default. 133.It Va net.inet.carp.suppress_preempt 134A read only value showing the status of preemption suppression. 135Preemption can be suppressed if link on an interface is down 136or when 137.Xr pfsync 4 138interface is not synchronized. 139Value of 0 means that preemption is not suppressed, since no 140problems are detected. 141Every problem increments suppression counter. 142.El 143.Sh ARP LEVEL LOAD BALANCING 144The 145.Nm 146has limited abilities for load balancing the incoming connections 147between hosts in Ethernet network. 148For load balancing operation, one needs several CARP interfaces that 149are configured to the same IP address, but to a different VHIDs. 150Once an ARP request is received, the CARP protocol will use a hashing 151function against the source IP address in the ARP request to determine 152which VHID should this request belong to. 153If the corresponding CARP interface is in master state, the ARP request 154will be replied, otherwise it will be ignored. 155See the 156.Sx EXAMPLES 157section for a practical example of load balancing. 158.Pp 159The ARP load balancing has some limitations. 160First, ARP balancing only works on the local network segment. 161It cannot balance traffic that crosses a router, because the 162router itself will always be balanced to the same virtual host. 163Second, ARP load balancing can lead to asymmetric routing 164of incoming and outgoing traffic, and thus combining it with 165.Xr pfsync 4 166is dangerous, because this creates a race condition between 167balanced routers and a host they are serving. 168Imagine an incoming packet creating state on the first router, being 169forwarded to its destination, and destination replying faster 170than the state information is packed and synced with the second router. 171If the reply would be load balanced to second router, it will be 172dropped due to no state. 173.Sh EXAMPLES 174For firewalls and routers with multiple interfaces, it is desirable to 175failover all of the 176.Nm 177interfaces together, when one of the physical interfaces goes down. 178This is achieved by the preempt option. 179Enable it on both host A and B: 180.Pp 181.Dl sysctl net.inet.carp.preempt=1 182.Pp 183Assume that host A is the preferred master and 192.168.1.x/24 is 184configured on one physical interface and 192.168.2.y/24 on another. 185This is the setup for host A: 186.Bd -literal -offset indent 187ifconfig carp0 create 188ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.1/24 189ifconfig carp1 create 190ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.2.1/24 191.Ed 192.Pp 193The setup for host B is identical, but it has a higher 194.Cm advskew : 195.Bd -literal -offset indent 196ifconfig carp0 create 197ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24 198ifconfig carp1 create 199ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24 200.Ed 201.Pp 202Because of the preempt option, when one of the physical interfaces of 203host A fails, 204.Cm advskew 205is adjusted to 240 on all its 206.Nm 207interfaces. 208This will cause host B to preempt on both interfaces instead of 209just the failed one. 210.Pp 211In order to set up an ARP balanced virtual host, it is necessary to configure 212one virtual host for each physical host which would respond to ARP requests 213and thus handle the traffic. 214In the following example, two virtual hosts are configured on two hosts to 215provide balancing and failover for the IP address 192.168.1.10. 216.Pp 217First the 218.Nm 219interfaces on host A are configured. 220The 221.Cm advskew 222of 100 on the second virtual host means that its advertisements will be sent 223out slightly less frequently. 224.Bd -literal -offset indent 225ifconfig carp0 create 226ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24 227ifconfig carp1 create 228ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24 229.Ed 230.Pp 231The configuration for host B is identical, except the 232.Cm advskew 233is on virtual host 1 rather than virtual host 2. 234.Bd -literal -offset indent 235ifconfig carp0 create 236ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24 237ifconfig carp1 create 238ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24 239.Ed 240.Pp 241Finally, the ARP balancing feature must be enabled on both hosts: 242.Pp 243.Dl sysctl net.inet.carp.arpbalance=1 244.Pp 245When the hosts receive an ARP request for 192.168.1.10, the source IP address 246of the request is used to compute which virtual host should answer the request. 247The host which is master of the selected virtual host will reply to the 248request, the other(s) will ignore it. 249.Pp 250This way, locally connected systems will receive different ARP replies and 251subsequent IP traffic will be balanced among the hosts. 252If one of the hosts fails, the other will take over the virtual MAC address, 253and begin answering ARP requests on its behalf. 254.Sh SEE ALSO 255.Xr inet 4 , 256.Xr pfsync 4 , 257.Xr rc.conf 5 , 258.Xr ifconfig 8 , 259.Xr sysctl 8 260.Sh HISTORY 261The 262.Nm 263device first appeared in 264.Ox 3.5 . 265It was imported into 266.Dx 1.11 . 267