1.\"- 2.\" Copyright (c) 2013 Gleb Smirnoff <glebius@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd March 22, 2017 29.Dt COUNTER 9 30.Os 31.Sh NAME 32.Nm counter 33.Nd "SMP-friendly kernel counter implementation" 34.Sh SYNOPSIS 35.In sys/types.h 36.In sys/systm.h 37.In sys/counter.h 38.Ft counter_u64_t 39.Fn counter_u64_alloc "int wait" 40.Ft void 41.Fn counter_u64_free "counter_u64_t c" 42.Ft void 43.Fn counter_u64_add "counter_u64_t c" "int64_t v" 44.Ft void 45.Fn counter_enter 46.Ft void 47.Fn counter_exit 48.Ft void 49.Fn counter_u64_add_protected "counter_u64_t c" "int64_t v" 50.Ft uint64_t 51.Fn counter_u64_fetch "counter_u64_t c" 52.Ft void 53.Fn counter_u64_zero "counter_u64_t c" 54.Ft int64_t 55.Fn counter_ratecheck "struct counter_rate *cr" "int64_t limit" 56.In sys/sysctl.h 57.Fn SYSCTL_COUNTER_U64 parent nbr name access ptr descr 58.Fn SYSCTL_ADD_COUNTER_U64 ctx parent nbr name access ptr descr 59.Fn SYSCTL_COUNTER_U64_ARRAY parent nbr name access ptr len descr 60.Fn SYSCTL_ADD_COUNTER_U64_ARRAY ctx parent nbr name access ptr len descr 61.Sh DESCRIPTION 62.Nm 63is a generic facility to create counters 64that can be utilized for any purpose (such as collecting statistical 65data). 66A 67.Nm 68is guaranteed to be lossless when several kernel threads do simultaneous 69updates. 70However, 71.Nm 72does not block the calling thread, 73also no 74.Xr atomic 9 75operations are used for the update, therefore the counters 76can be used in any non-interrupt context. 77Moreover, 78.Nm 79has special optimisations for SMP environments, making 80.Nm 81update faster than simple arithmetic on the global variable. 82Thus 83.Nm 84is considered suitable for accounting in the performance-critical 85code paths. 86.Bl -tag -width indent 87.It Fn counter_u64_alloc wait 88Allocate a new 64-bit unsigned counter. 89The 90.Fa wait 91argument is the 92.Xr malloc 9 93wait flag, should be either 94.Va M_NOWAIT 95or 96.Va M_WAITOK . 97If 98.Va M_NOWAIT 99is specified the operation may fail. 100.It Fn counter_u64_free c 101Free the previously allocated counter 102.Fa c . 103.It Fn counter_u64_add c v 104Add 105.Fa v 106to 107.Fa c . 108The KPI does not guarantee any protection from wraparound. 109.It Fn counter_enter 110Enter mode that would allow the safe update of several counters via 111.Fn counter_u64_add_protected . 112On some machines this expands to 113.Xr critical 9 114section, while on other is a nop. 115See 116.Sx IMPLEMENTATION DETAILS . 117.It Fn counter_exit 118Exit mode for updating several counters. 119.It Fn counter_u64_add_protected c v 120Same as 121.Fn counter_u64_add , 122but should be preceded by 123.Fn counter_enter . 124.It Fn counter_u64_fetch c 125Take a snapshot of counter 126.Fa c . 127The data obtained is not guaranteed to reflect the real cumulative 128value for any moment. 129.It Fn counter_u64_zero c 130Clear the counter 131.Fa c 132and set it to zero. 133.It Fn counter_ratecheck cr limit 134The function is a multiprocessor-friendly version of 135.Fn ppsratecheck 136which uses 137.Nm 138internally. 139Returns non-negative value if the rate is not yet reached during the current 140second, and a negative value otherwise. 141If the limit was reached on previous second, but was just reset back to zero, 142then 143.Fn counter_ratecheck 144returns number of events since previous reset. 145.It Fn SYSCTL_COUNTER_U64 parent nbr name access ptr descr 146Declare a static 147.Xr sysctl 9 148oid that would represent a 149.Nm . 150The 151.Fa ptr 152argument should be a pointer to allocated 153.Vt counter_u64_t . 154A read of the oid returns value obtained through 155.Fn counter_u64_fetch . 156Any write to the oid zeroes it. 157.It Fn SYSCTL_ADD_COUNTER_U64 ctx parent nbr name access ptr descr 158Create a 159.Xr sysctl 9 160oid that would represent a 161.Nm . 162The 163.Fa ptr 164argument should be a pointer to allocated 165.Vt counter_u64_t . 166A read of the oid returns value obtained through 167.Fn counter_u64_fetch . 168Any write to the oid zeroes it. 169.It Fn SYSCTL_COUNTER_U64_ARRAY parent nbr name access ptr len descr 170Declare a static 171.Xr sysctl 9 172oid that would represent an array of 173.Nm . 174The 175.Fa ptr 176argument should be a pointer to allocated array of 177.Vt counter_u64_t's . 178The 179.Fa len 180argument should specify number of elements in the array. 181A read of the oid returns len-sized array of 182.Vt uint64_t 183values obtained through 184.Fn counter_u64_fetch . 185Any write to the oid zeroes all array elements. 186.It Fn SYSCTL_ADD_COUNTER_U64_ARRAY ctx parent nbr name access ptr len descr 187Create a 188.Xr sysctl 9 189oid that would represent an array of 190.Nm . 191The 192.Fa ptr 193argument should be a pointer to allocated array of 194.Vt counter_u64_t's . 195The 196.Fa len 197argument should specify number of elements in the array. 198A read of the oid returns len-sized array of 199.Vt uint64_t 200values obtained through 201.Fn counter_u64_fetch . 202Any write to the oid zeroes all array elements. 203.El 204.Sh IMPLEMENTATION DETAILS 205On all architectures 206.Nm 207is implemented using per-CPU data fields that are specially aligned 208in memory, to avoid inter-CPU bus traffic due to shared use 209of the variables between CPUs. 210These are allocated using 211.Va UMA_ZONE_PCPU 212.Xr uma 9 213zone. 214The update operation only touches the field that is private to current CPU. 215Fetch operation loops through all per-CPU fields and obtains a snapshot 216sum of all fields. 217.Pp 218On amd64 a 219.Nm counter 220update is implemented as a single instruction without lock semantics, 221operating on the private data for the current CPU, 222which is safe against preemption and interrupts. 223.Pp 224On i386 architecture, when machine supports the cmpxchg8 instruction, 225this instruction is used. 226The multi-instruction sequence provides the same guarantees as the 227amd64 single-instruction implementation. 228.Pp 229On some architectures updating a counter require a 230.Xr critical 9 231section. 232.Sh EXAMPLES 233The following example creates a static counter array exported to 234userspace through a sysctl: 235.Bd -literal -offset indent 236#define MY_SIZE 8 237static counter_u64_t array[MY_SIZE]; 238SYSCTL_COUNTER_U64_ARRAY(_debug, OID_AUTO, counter_array, CTLFLAG_RW, 239 &array[0], MY_SIZE, "Test counter array"); 240.Ed 241.Sh SEE ALSO 242.Xr atomic 9 , 243.Xr critical 9 , 244.Xr locking 9 , 245.Xr malloc 9 , 246.Xr ratecheck 9 , 247.Xr sysctl 9 , 248.Xr uma 9 249.Sh HISTORY 250The 251.Nm 252facility first appeared in 253.Fx 10.0 . 254.Sh AUTHORS 255.An -nosplit 256The 257.Nm 258facility was written by 259.An Gleb Smirnoff 260and 261.An Konstantin Belousov . 262