1.\" Copyright (c) 2012-2013 Luigi Rizzo, Universita` di Pisa 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" This document is derived in part from the enet man page (enet.4) 26.\" distributed with 4.3BSD Unix. 27.\" 28.\" $FreeBSD: head/share/man/man4/vale.4 228017 2011-11-27 06:55:57Z gjb $ 29.\" 30.Dd December 26, 2013 31.Dt VALE 4 32.Os 33.Sh NAME 34.Nm vale 35.Nd a very fast Virtual Local Ethernet using the netmap API 36.Sh SYNOPSIS 37.Cd device netmap 38.Sh DESCRIPTION 39.Nm 40is a feature of the 41.Xr netmap 4 42module that implements multiple virtual switches that can 43be used to interconnect 44.Xr netmap 4 45clients, including traffic sources and sinks, packet forwarders, 46userspace firewalls, and so on. 47.Pp 48.Nm 49is implemented completely in software, and is extremely fast. 50On a modern machine it can move almost 20 Million packets per 51second (Mpps) per core with small frames, and about 70 Gbit/s 52with 1500 byte frames. 53.Sh OPERATION 54.Nm 55dynamically creates switches and ports as client connect 56to it using the 57.Xr netmap 4 58API. 59.Pp 60.Nm 61ports are named 62.Pa vale[bdg:][port] 63where 64.Pa vale 65is the prefix indicating a VALE switch rather than a standard interface, 66.Pa bdg 67indicates a specific switch (the colon is a separator), 68and 69.Pa port 70indicates a port within the switch. 71Bridge and ports names are arbitrary strings, the only 72constraint being that the full name must fit within 16 73characters. 74.Pp 75.Nm 76ports can be physical network interfaces that support the 77.Xr netmap 4 78API 79by specifying the interface name for 80.Pa [port] . 81See 82.Xr netmap 4 83for details of the naming rule. 84.Pp 85Physical interfaces are attached using the 86.Dv NIOCGREGIF 87command of 88.Xr ioctl 2 , 89and 90.Dv NETMAP_BDG_ATTACH 91for the 92.Va nr_cmd 93field in 94.Vt struct nmreq . 95The corresponding host stack can also be attached to the bridge, specifying 96.Dv NETMAP_BDG_HOST 97in 98.Va nr_arg1 . 99To detach the interface from the bridge, 100.Dv NETMAP_BDG_DETACH 101is used instead of 102.Dv NETMAP_BDG_ATTACH . 103The host stack is also detached from the bridge at the same 104time if it had been attached. 105.Pp 106Physical interfaces are treated as a system configuration; 107they remain attached even after the configuring process died, 108and can be detached by any other process. 109.Pp 110Once a physical interface is attached, this interface is no longer 111available to be directly accessed by 112.Xr netmap 4 113clients (user processes) or to be attached by another bridge. 114On the other hand, when a 115.Xr netmap 4 116client holds the physical interface, 117this interface cannot be attached to a bridge. 118.Pp 119.Va NETMAP_BDG_LIST 120subcommand in 121.Va nr_cmd 122of 123.Vt struct nmreq 124is used to obtain bridge and port information. 125There are two modes of how this works. 126If any 127.Va nr_name 128starting from non '\\0' is provided, 129.Xr ioctl 2 130returning indicates the position of the named interface. 131This position is represented by an index of the bridge and the port, 132and put into the 133.Va nr_arg1 134and 135.Va nr_arg2 136fields, respectively. 137If the named interface does not exist, 138.Xr ioctl 2 139returns 140.Dv EINVAL . 141.Pp 142If 143.Va nr_name 144starting from '\\0' is provided, 145.Xr ioctl 2 146returning indicates the 147first existing interface on and after the position specified in 148.Va nr_arg1 149and 150.Va nr_arg2 . 151If the caller specified a port index greater than the highest 152index of the ports, it is recognized as port index 0 of the 153next bridge ( 154.Va nr_arg1 155+ 1, 156.Va nr_arg2 157= 0). 158.Xr ioctl 2 159returns 160.Dv EINVAL 161if the given position is higher than that of 162any existing interface. 163On successful return of 164.Xr ioctl 2 , 165the interface name is also stored in 166.Va nr_name . 167.Dv NETMAP_BDG_LIST 168is always used with the 169.Dv NIOCGINFO 170command of 171.Xr ioctl 2 . 172.Pp 173Below is an example of printing all the existing ports walking through 174all the bridges. 175.Bd -literal 176struct nmreq nmr; 177int fd = open("/dev/netmap", O_RDWR); 178 179bzero(&nmr, sizeof(nmr)); 180nmr.nr_version = NETMAP_API; 181nmr.nr_cmd = NETMAP_BDG_LIST; 182nmr.nr_arg1 = nmr.nr_arg2 = 0; /* start from bridge:0 port:0 */ 183for (; !ioctl(fd, NIOCGINFO, &nmr); nmr.nr_arg2++) { 184 D("bridge:%d port:%d %s", nmr.nr_arg1, nmr.nr_arg2, 185 nmr.nr_name); 186 nmr.nr_name[0] = '\\0'; 187} 188.Ed 189.Ss LIMITS 190.Nm 191currently supports up to 8 switches, 254 ports per switch, 1921024 buffers per port. 193These hard limits will be changed to 194.Xr sysctl 8 195variables in future releases. 196.Pp 197Attaching the host stack to the bridge imposes significant performance 198degradation when many packets are forwarded to the host stack. 199This is because each packet forwarded to the host stack causes 200.Xr mbuf 9 201allocation in the same thread context. 202.Sh SYSCTL VARIABLES 203.Nm 204uses the following 205.Xr sysctl 8 206variables to control operation: 207.Bl -tag -width "dev.netmap.verbose" 208.It dev.netmap.bridge 209The maximum number of packets processed internally 210in each iteration. 211Defaults to 1024, use lower values to trade latency 212with throughput. 213.It dev.netmap.verbose 214Set to non-zero values to enable in-kernel diagnostics. 215.El 216.Sh EXAMPLES 217Create one switch, with a traffic generator connected to one 218port, and a 219.Xr netmap 4 Ns -enabled 220.Xr tcpdump 1 221instance on another port: 222.Bd -literal -offset indent 223tcpdump -ni vale-a:1 & 224pkt-gen -i vale-a:0 -f tx & 225.Ed 226.Pp 227Create two switches, 228each connected to two qemu machines on different ports. 229.Bd -literal -offset indent 230qemu -net nic -net netmap,ifname=vale-1:a ... & 231qemu -net nic -net netmap,ifname=vale-1:b ... & 232qemu -net nic -net netmap,ifname=vale-2:c ... & 233qemu -net nic -net netmap,ifname=vale-2:d ... & 234.Ed 235.Sh SEE ALSO 236.Xr netmap 4 237.Rs 238.%A Luigi Rizzo 239.%A Giuseppe Lettieri 240.%T VALE, a switched ethernet for virtual machines 241.%J http://info.iet.unipi.it/~luigi/vale/ 242.%D June 2012 243.Re 244.Sh AUTHORS 245.An -nosplit 246The 247.Nm 248switch has been designed and implemented in 2012 by 249.An Luigi Rizzo 250and 251.An Giuseppe Lettieri 252at the Universita` di Pisa. 253.Pp 254.Nm 255has been funded by the European Commission within the FP7 Projects 256CHANGE (257422) and OPENLAB (287581). 257