1.\" Copyright (c) 2015-2017 Amazon.com, Inc. or its affiliates. 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in 13.\" the documentation and/or other materials provided with the 14.\" distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 17.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 18.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 19.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 20.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 21.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 22.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 23.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 24.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 25.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27.\" 28.\" $FreeBSD$ 29.\" 30.Dd August 16, 2017 31.Dt ENA 4 32.Os 33.Sh NAME 34.Nm ena 35.Nd "FreeBSD kernel driver for Elastic Network Adapter (ENA) family" 36.Sh SYNOPSIS 37To compile this driver into the kernel, 38place the following line in the 39kernel configuration file: 40.Bd -ragged -offset indent 41.Cd "device ena" 42.Ed 43.Pp 44Alternatively, to load the driver as a 45module at boot time, place the following line in 46.Xr loader.conf 5 : 47.Bd -literal -offset indent 48if_ena_load="YES" 49.Ed 50.Sh DESCRIPTION 51The ENA is a networking interface designed to make good use of modern CPU 52features and system architectures. 53.Pp 54The ENA device exposes a lightweight management interface with a 55minimal set of memory mapped registers and extendable command set 56through an Admin Queue. 57.Pp 58The driver supports a range of ENA devices, is link-speed independent 59(i.e., the same driver is used for 10GbE, 25GbE, 40GbE, etc.), and has 60a negotiated and extendable feature set. 61.Pp 62Some ENA devices support SR-IOV. 63This driver is used for both the SR-IOV Physical Function (PF) and Virtual 64Function (VF) devices. 65.Pp 66The ENA devices enable high speed and low overhead network traffic 67processing by providing multiple Tx/Rx queue pairs (the maximum number 68is advertised by the device via the Admin Queue), a dedicated MSI-X 69interrupt vector per Tx/Rx queue pair, and CPU cacheline optimized 70data placement. 71.Pp 72The 73.Nm 74driver supports industry standard TCP/IP offload features such 75as checksum offload and TCP transmit segmentation offload (TSO). 76Receive-side scaling (RSS) is supported for multi-core scaling. 77.Pp 78The 79.Nm 80driver and its corresponding devices implement health 81monitoring mechanisms such as watchdog, enabling the device and driver 82to recover in a manner transparent to the application, as well as 83debug logs. 84.Pp 85Some of the ENA devices support a working mode called Low-latency 86Queue (LLQ), which saves several more microseconds. 87This feature will be implemented for driver in future releases. 88.Sh HARDWARE 89Supported PCI vendor ID/device IDs: 90.Pp 91.Bl -bullet -compact 92.It 931d0f:0ec2 - ENA PF 94.It 951d0f:1ec2 - ENA PF with LLQ support 96.It 971d0f:ec20 - ENA VF 98.It 991d0f:ec21 - ENA VF with LLQ support 100.El 101.Sh DIAGNOSTICS 102.Ss Device initialization phase: 103.Bl -diag 104.It ena%d: failed to init mmio read less 105.Pp 106Error occurred during initialization of the mmio register read request. 107.It ena%d: Can not reset device 108.Pp 109Device could not be reset. 110.br 111Device may not be responding or is already during reset. 112.It ena%d: device version is too low 113.Pp 114Version of the controller is too old and it is not supported by the driver. 115.It ena%d: Invalid dma width value %d 116.Pp 117The controller is able to request dma transaction width. 118.br 119Device stopped responding or it demanded invalid value. 120.It ena%d: Can not initialize ena admin queue with device 121.Pp 122Initialization of the Admin Queue failed. 123.br 124Device may not be responding or there was a problem with initialization of 125the resources. 126.It ena%d: Cannot get attribute for ena device rc: %d 127.Pp 128Failed to get attributes of the device from the controller. 129.It ena%d: Cannot configure aenq groups rc: %d 130.Pp 131Errors occurred when trying to configure AENQ groups. 132.El 133.Ss Driver initialisation/shutdown phase: 134.Bl -diag 135.It ena%d: PCI resource allocation failed! 136.It ena%d: allocating ena_dev failed 137.It ena%d: failed to pmap registers bar 138.It ena%d: Error while setting up bufring 139.It ena%d: Error with initialization of IO rings 140.It ena%d: can not allocate ifnet structure 141.It ena%d: Error with network interface setup 142.It ena%d: Failed to enable and set the admin interrupts 143.It ena%d: Failed to allocate %d, vectors %d 144.It ena%d: Failed to enable MSIX, vectors %d rc %d 145.It ena%d: Error with MSI-X enablement 146.It ena%d: could not allocate irq vector: %d 147.It ena%d: Unable to allocate bus resource: registers 148.Pp 149Resource allocation failed when initializing the device. 150.br 151Driver will not be attached. 152.It ena%d: ENA device init failed (err: %d) 153.Pp 154Device initialization failed. 155.br 156Driver will not be attached. 157.It ena%d: could not activate irq vector: %d 158.Pp 159Error occurred when trying to activate interrupt vectors for Admin Queue. 160.It ena%d: failed to register interrupt handler for irq %ju: %d 161.Pp 162Error occurred when trying to register Admin Queue interrupt handler. 163.It ena%d: Cannot setup mgmnt queue intr 164.Pp 165Error occurred during configuration of the Admin Queue interrupts. 166.It ena%d: Enable MSI-X failed 167.Pp 168Configuration of the MSI-X for Admin Queue failed. 169.br 170There could be lack of resources or interrupts could not have been configured. 171.br 172Driver will not be attached. 173.It ena%d: VLAN is in use, detach first 174.Pp 175VLANs are being used when trying to detach the driver. 176.br 177VLANs must be detached first and then detach routine have to be called again. 178.It ena%d: Unmapped RX DMA tag associations 179.It ena%d: Unmapped TX DMA tag associations 180.Pp 181Error occurred when trying to destroy RX/TX DMA tag. 182.It ena%d: Cannot init RSS 183.It ena%d: Cannot fill indirect table 184.It ena%d: Cannot fill indirect table 185.It ena%d: Cannot fill hash function 186.It ena%d: Cannot fill hash control 187.It ena%d: WARNING: RSS was not properly initialized, it will affect bandwidth 188.Pp 189Error occurred during initialization of one of RSS resources. 190.br 191The device will work with reduced performance because all RX packets will be 192passed to queue 0 and there will be no hash information. 193.It ena%d: failed to tear down irq: %d 194.It ena%d: dev has no parent while releasing res for irq: %d 195Release of the interrupts failed. 196.El 197.Ss Additional diagnostic: 198.Bl -diag 199.It ena%d: Cannot get attribute for ena device 200.Pp 201This message appears when trying to change MTU and driver is unable to get 202attributes from the device. 203.It ena%d: Invalid MTU setting. new_mtu: %d 204.Pp 205Requested MTU value is not supported and will not be set. 206.It ena%d: keep alive watchdog timeout 207.Pp 208Device stopped responding and will be reset. 209.It ena%d: Found a Tx that wasn't completed on time, qid %d, index %d. 210.Pp 211Packet was pushed to the NIC but not sent within given time limit. 212.br 213It may be caused by hang of the IO queue. 214.It ena%d: The number of lost tx completion is aboce the threshold (%d > %d). Reset the device 215.Pp 216If too many Tx wasn't completed on time the device is going to be reset. 217.br 218It may be caused by hanged queue or device. 219.It ena%d: trigger reset is on 220.Pp 221Device will be reset. 222.br 223Reset is triggered either by watchdog or if too many TX packets were not 224completed on time. 225.It ena%d: invalid value recvd 226.Pp 227Link status received from the device in the AENQ handler is invalid. 228.It ena%d: Allocation for Tx Queue %u failed 229.It ena%d: Allocation for Rx Queue %u failed 230.It ena%d: Unable to create Rx DMA map for buffer %d 231.It ena%d: Failed to create io TX queue #%d rc: %d 232.It ena%d: Failed to get TX queue handlers. TX queue num %d rc: %d 233.It ena%d: Failed to create io RX queue[%d] rc: %d 234.It ena%d: Failed to get RX queue handlers. RX queue num %d rc: %d 235.It ena%d: failed to request irq 236.It ena%d: could not allocate irq vector: %d 237.It ena%d: failed to register interrupt handler for irq %ju: %d 238.Pp 239IO resources initialization failed. 240.br 241Interface will not be brought up. 242.It ena%d: LRO[%d] Initialization failed! 243.Pp 244Initialization of the LRO for the RX ring failed. 245.It ena%d: failed to alloc buffer for rx queue 246.It ena%d: failed to add buffer for rx queue %d 247.It ena%d: refilled rx queue %d with %d pages only 248.Pp 249Allocation of resources used on RX path failed. 250.br 251If happened during initialization of the IO queue, the interface will not be 252brought up. 253.It ena%d: ioctl promisc/allmulti 254.Pp 255IOCTL request for the device to work in promiscuous/allmulti mode. 256.br 257See 258.Xr ifconfig 8 259for more details. 260.It ena%d: too many fragments. Last fragment: %d! 261.Pp 262Packet with unsupported number of segments was queued for sending to the 263device. 264.br 265Packet will be dropped. 266.El 267.Sh SUPPORT 268If an issue is identified with the released source code with a supported 269adapter, please email the specific information related to the issue to 270.Aq Mt mk@semihalf.com 271and 272.Aq Mt mw@semihalf.com . 273.Sh SEE ALSO 274.Xr vlan 4 , 275.Xr ifconfig 8 276.Sh AUTHORS 277The 278.Nm 279driver was written by 280.An Semihalf . 281