1.\"- 2.\" Copyright (c) 2017 Robert N. M. Watson 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd July 5, 2018 29.Dt DPCPU 9 30.Os 31.Sh NAME 32.Nm dpcpu 33.Nd Kernel Dynamic Per-CPU Memory Allocator 34.Sh SYNOPSIS 35.In sys/pcpu.h 36.Ss Per-CPU Variable Definition and Declaration 37.Fn DPCPU_DEFINE "type" "name" 38.Fn DPCPU_DEFINE_STATIC "type" "name" 39.Fn DPCPU_DECLARE "type" "name" 40.Ss Current CPU Accessor Functions 41.Fn DPCPU_PTR "name" 42.Fn DPCPU_GET "name" 43.Fn DPCPU_SET "name" "value" 44.Ss Named CPU Accessor Functions 45.Fn DPCPU_ID_PTR "cpu" "name" 46.Fn DPCPU_ID_GET "cpu" "name" 47.Fn DPCPU_ID_SET "cpu" "name" "value" 48.Sh DESCRIPTION 49.Nm 50instantiates one instance of a global variable with each CPU in the system. 51Dynamically allocated per-CPU variables are defined using 52.Fn DPCPU_DEFINE , 53which defines a variable of name 54.Ar name 55and type 56.Ar type . 57Arbitrary C types may be used, including structures and arrays. 58If no initialization is provided, then each per-CPU instance of the variable 59will be zero-filled (i.e., as though allocated in BSS): 60.Bd -literal -offset 1234 61DPCPU_DEFINE(int, foo_int); 62.Ed 63.Pp 64Values may also be initialized statically with the definition, causing each 65per-CPU instance to be initialized with the value: 66.Bd -literal -offset 1234 67DPCPU_DEFINE(int, foo_int) = 1; 68.Ed 69.Pp 70Values that can be defined as 71.Dv static 72must use 73.Fn DPCPU_DEFINE_STATIC : 74.Bd -literal -offset 1234 75DPCPU_DEFINE_STATIC(int, foo_int); 76.Ed 77.Pp 78.Fn DPCPU_DECLARE 79produces a declaration of the per-CPU variable suitable for use in header 80files. 81.Pp 82The current CPU's variable instance can be accessed via 83.Nm DPCPU_PTR 84(which returns a pointer to the per-CPU instance), 85.Nm DPCPU_GET 86(which retrieves the value of the per-CPU instance), 87and 88.Nm DPCPU_SET 89(which sets the value of the per-CPU instance). 90.Pp 91Instances of variables associated with specific CPUs can be accessed via the 92.Nm DPCPU_ID_PTR , 93.Nm DPCPU_ID_GET , 94and 95.Nm DPGPU_ID_SET 96accessor functions, which accept an additional CPU ID argument, 97.Ar cpu . 98.Ss Synchronization 99In addition to the ordinary synchronization concerns associated with global 100variables, which may imply the use of 101.Xr atomic 9 , 102.Xr mutex 9 , 103or other kernel synchronization primitives, it is further the case that 104thread migration could dynamically change the instance of a variable being 105accessed by a thread between operations. 106This requires additional care when reasoning about and protecting per-CPU 107variables. 108.Pp 109For example, it may be desirable to protect access using 110.Xr critical_section 9 111to prevent both preemption and migration during use. 112Alternatively, it may be desirable to cache the CPU ID at the start of a 113sequence of accesses, using suitable synchronization to make non-atomic 114sequences safe in the presence of migration. 115.Bd -literal -offset 1234 116DPCPU_DEFINE_STATIC(int, foo_int); 117DPCPU_DEFINE_STATIC(struct mutex, foo_lock); 118 119void 120foo_int_increment(void) 121{ 122 int cpu, value; 123 124 /* Safe as atomic access. */ 125 atomic_add_int(DPCPU_PTR(foo_int), 1); 126 127 /* 128 * Protect with a critical section, which prevents preemption 129 * and migration. However, access to instances from remote CPUs 130 * is not safe, as critical sections prevent concurrent access 131 * only from the current CPU. 132 */ 133 critical_enter(); 134 value = DPCPU_GET(foo_int); 135 value++; 136 DPCPU_SET(foo_int, value); 137 critical_exit(); 138 139 /* 140 * Protect with a per-CPU mutex, tolerating migration, but 141 * potentially accessing the variable from multiple CPUs if 142 * migration occurs after reading curcpu. Remote access to a 143 * per-CPU variable is safe as long as the correct mutex is 144 * acquired. 145 */ 146 cpu = curcpu; 147 mtx_lock(DPCPU_ID_PTR(cpu, foo_lock)); 148 value = DPCPU_ID_GET(cpu, foo_int); 149 value++; 150 DPCPU_ID_SET(cpu, foo_int); 151 mtx_unlock(DPCPU_ID_PTR(cpu, foo_lock)); 152} 153.Ed 154.Sh SEE ALSO 155.Xr atomic 9 , 156.Xr critical_enter 9 , 157.Xr mutex 9 158.Sh HISTORY 159.Nm 160was first introduced by 161.An Jeff Roberson 162in 163.Fx 8.0 . 164This manual page was written by 165.An Robert N. M. Watson. 166