1.. SPDX-License-Identifier: GPL-2.0
2
3====================
4Interface statistics
5====================
6
7Overview
8========
9
10This document is a guide to Linux network interface statistics.
11
12There are three main sources of interface statistics in Linux:
13
14 - standard interface statistics based on
15   :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`;
16 - protocol-specific statistics; and
17 - driver-defined statistics available via ethtool.
18
19Standard interface statistics
20-----------------------------
21
22There are multiple interfaces to reach the standard statistics.
23Most commonly used is the `ip` command from `iproute2`::
24
25  $ ip -s -s link show dev ens4u1u1
26  6: ens4u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
27    link/ether 48:2a:e3:4c:b1:d1 brd ff:ff:ff:ff:ff:ff
28    RX: bytes  packets  errors  dropped overrun mcast
29    74327665117 69016965 0       0       0       0
30    RX errors: length   crc     frame   fifo    missed
31               0        0       0       0       0
32    TX: bytes  packets  errors  dropped carrier collsns
33    21405556176 44608960 0       0       0       0
34    TX errors: aborted  fifo   window heartbeat transns
35               0        0       0       0       128
36    altname enp58s0u1u1
37
38Note that `-s` has been specified twice to see all members of
39:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
40If `-s` is specified once the detailed errors won't be shown.
41
42`ip` supports JSON formatting via the `-j` option.
43
44Queue statistics
45~~~~~~~~~~~~~~~~
46
47Queue statistics are accessible via the netdev netlink family.
48
49Currently no widely distributed CLI exists to access those statistics.
50Kernel development tools (ynl) can be used to experiment with them,
51see `Documentation/userspace-api/netlink/intro-specs.rst`.
52
53Protocol-specific statistics
54----------------------------
55
56Protocol-specific statistics are exposed via relevant interfaces,
57the same interfaces as are used to configure them.
58
59ethtool
60~~~~~~~
61
62Ethtool exposes common low-level statistics.
63All the standard statistics are expected to be maintained
64by the device, not the driver (as opposed to driver-defined stats
65described in the next section which mix software and hardware stats).
66For devices which contain unmanaged
67switches (e.g. legacy SR-IOV or multi-host NICs) the events counted
68may not pertain exclusively to the packets destined to
69the local host interface. In other words the events may
70be counted at the network port (MAC/PHY blocks) without separation
71for different host side (PCIe) devices. Such ambiguity must not
72be present when internal switch is managed by Linux (so called
73switchdev mode for NICs).
74
75Standard ethtool statistics can be accessed via the interfaces used
76for configuration. For example ethtool interface used
77to configure pause frames can report corresponding hardware counters::
78
79  $ ethtool --include-statistics -a eth0
80  Pause parameters for eth0:
81  Autonegotiate:	on
82  RX:			on
83  TX:			on
84  Statistics:
85    tx_pause_frames: 1
86    rx_pause_frames: 1
87
88General Ethernet statistics not associated with any particular
89functionality are exposed via ``ethtool -S $ifc`` by specifying
90the ``--groups`` parameter::
91
92  $ ethtool -S eth0 --groups eth-phy eth-mac eth-ctrl rmon
93  Stats for eth0:
94  eth-phy-SymbolErrorDuringCarrier: 0
95  eth-mac-FramesTransmittedOK: 1
96  eth-mac-FrameTooLongErrors: 1
97  eth-ctrl-MACControlFramesTransmitted: 1
98  eth-ctrl-MACControlFramesReceived: 0
99  eth-ctrl-UnsupportedOpcodesReceived: 1
100  rmon-etherStatsUndersizePkts: 1
101  rmon-etherStatsJabbers: 0
102  rmon-rx-etherStatsPkts64Octets: 1
103  rmon-rx-etherStatsPkts65to127Octets: 0
104  rmon-rx-etherStatsPkts128to255Octets: 0
105  rmon-tx-etherStatsPkts64Octets: 2
106  rmon-tx-etherStatsPkts65to127Octets: 3
107  rmon-tx-etherStatsPkts128to255Octets: 0
108
109Driver-defined statistics
110-------------------------
111
112Driver-defined ethtool statistics can be dumped using `ethtool -S $ifc`, e.g.::
113
114  $ ethtool -S ens4u1u1
115  NIC statistics:
116     tx_single_collisions: 0
117     tx_multi_collisions: 0
118
119uAPIs
120=====
121
122procfs
123------
124
125The historical `/proc/net/dev` text interface gives access to the list
126of interfaces as well as their statistics.
127
128Note that even though this interface is using
129:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
130internally it combines some of the fields.
131
132sysfs
133-----
134
135Each device directory in sysfs contains a `statistics` directory (e.g.
136`/sys/class/net/lo/statistics/`) with files corresponding to
137members of :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
138
139This simple interface is convenient especially in constrained/embedded
140environments without access to tools. However, it's inefficient when
141reading multiple stats as it internally performs a full dump of
142:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
143and reports only the stat corresponding to the accessed file.
144
145Sysfs files are documented in
146`Documentation/ABI/testing/sysfs-class-net-statistics`.
147
148
149netlink
150-------
151
152`rtnetlink` (`NETLINK_ROUTE`) is the preferred method of accessing
153:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` stats.
154
155Statistics are reported both in the responses to link information
156requests (`RTM_GETLINK`) and statistic requests (`RTM_GETSTATS`,
157when `IFLA_STATS_LINK_64` bit is set in the `.filter_mask` of the request).
158
159netdev (netlink)
160~~~~~~~~~~~~~~~~
161
162`netdev` generic netlink family allows accessing page pool and per queue
163statistics.
164
165ethtool
166-------
167
168Ethtool IOCTL interface allows drivers to report implementation
169specific statistics. Historically it has also been used to report
170statistics for which other APIs did not exist, like per-device-queue
171statistics, or standard-based statistics (e.g. RFC 2863).
172
173Statistics and their string identifiers are retrieved separately.
174Identifiers via `ETHTOOL_GSTRINGS` with `string_set` set to `ETH_SS_STATS`,
175and values via `ETHTOOL_GSTATS`. User space should use `ETHTOOL_GDRVINFO`
176to retrieve the number of statistics (`.n_stats`).
177
178ethtool-netlink
179---------------
180
181Ethtool netlink is a replacement for the older IOCTL interface.
182
183Protocol-related statistics can be requested in get commands by setting
184the `ETHTOOL_FLAG_STATS` flag in `ETHTOOL_A_HEADER_FLAGS`. Currently
185statistics are supported in the following commands:
186
187  - `ETHTOOL_MSG_PAUSE_GET`
188  - `ETHTOOL_MSG_FEC_GET`
189  - `ETHTOOL_MSG_MM_GET`
190
191debugfs
192-------
193
194Some drivers expose extra statistics via `debugfs`.
195
196struct rtnl_link_stats64
197========================
198
199.. kernel-doc:: include/uapi/linux/if_link.h
200    :identifiers: rtnl_link_stats64
201
202Notes for driver authors
203========================
204
205Drivers should report all statistics which have a matching member in
206:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` exclusively
207via `.ndo_get_stats64`. Reporting such standard stats via ethtool
208or debugfs will not be accepted.
209
210Drivers must ensure best possible compliance with
211:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
212Please note for example that detailed error statistics must be
213added into the general `rx_error` / `tx_error` counters.
214
215The `.ndo_get_stats64` callback can not sleep because of accesses
216via `/proc/net/dev`. If driver may sleep when retrieving the statistics
217from the device it should do so periodically asynchronously and only return
218a recent copy from `.ndo_get_stats64`. Ethtool interrupt coalescing interface
219allows setting the frequency of refreshing statistics, if needed.
220
221Retrieving ethtool statistics is a multi-syscall process, drivers are advised
222to keep the number of statistics constant to avoid race conditions with
223user space trying to read them.
224
225Statistics must persist across routine operations like bringing the interface
226down and up.
227
228Kernel-internal data structures
229-------------------------------
230
231The following structures are internal to the kernel, their members are
232translated to netlink attributes when dumped. Drivers must not overwrite
233the statistics they don't report with 0.
234
235- ethtool_pause_stats()
236- ethtool_fec_stats()
237