1.\" Copyright (c) 2000-2001 John H. Baldwin <jhb@FreeBSD.org> 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE DEVELOPERS ``AS IS'' AND ANY EXPRESS OR 14.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 15.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 16.\" IN NO EVENT SHALL THE DEVELOPERS BE LIABLE FOR ANY DIRECT, INDIRECT, 17.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 18.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 19.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 20.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 21.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 22.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23.\" 24.\" $FreeBSD: src/share/man/man9/atomic.9,v 1.17 2010/05/27 13:56:27 uqs Exp $ 25.\" 26.Dd January 14, 2024 27.Dt ATOMIC 9 28.Os 29.Sh NAME 30.Nm atomic_add , 31.Nm atomic_clear , 32.Nm atomic_cmpset , 33.Nm atomic_fetchadd , 34.Nm atomic_load , 35.Nm atomic_readandclear , 36.Nm atomic_set , 37.Nm atomic_subtract , 38.Nm atomic_store 39.Nd atomic operations 40.Sh SYNOPSIS 41.In sys/types.h 42.In machine/atomic.h 43.Ft void 44.Fn atomic_add_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 45.Ft void 46.Fn atomic_clear_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 47.Ft int 48.Fo atomic_cmpset_[acq_|rel_]<type> 49.Fa "volatile <type> *dst" 50.Fa "<type> old" 51.Fa "<type> new" 52.Fc 53.Ft <type> 54.Fn atomic_fetchadd_<type> "volatile <type> *p" "<type> v" 55.Ft <type> 56.Fn atomic_load_[acq_]<type> "volatile <type> *p" 57.Ft <type> 58.Fn atomic_readandclear_<type> "volatile <type> *p" 59.Ft void 60.Fn atomic_set_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 61.Ft void 62.Fn atomic_subtract_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 63.Ft void 64.Fn atomic_store_[rel_]<type> "volatile <type> *p" "<type> v" 65.Sh DESCRIPTION 66Each of the atomic operations is guaranteed to be atomic in the presence of 67interrupts. 68They can be used to implement reference counts or as building blocks for more 69advanced synchronization primitives, such as mutexes. 70.Pp 71On all architectures supported by 72. Dx , 73ordinary loads and stores of integers in cache-coherent memory are 74inherently atomic if the integer is naturally aligned and its size does 75not exceed the processor's word size. 76However, such loads and stores may be elided from the program by the 77compiler, whereas atomic operations are always performed. 78.Pp 79Except as noted below, the semantics of these operations are almost 80identical to the semantics of similarly named C11 atomic operations. 81.Ss Types 82Each atomic operation operates on a specific 83.Fa type . 84The type to use is indicated in the function name. 85In contrast to C11 atomic operations, 86.Dx Ns 's 87atomic operations are performed on ordinary integer types. 88The available types are: 89.Pp 90.Bl -tag -offset indent -width ".Li cpumask" -compact 91.It Li cpumask 92CPU mask (cpumask_t) 93.It Li int 94unsigned integer 95.It Li long 96unsigned long integer 97.It Li ptr 98unsigned integer the size of a pointer 99.It Li 32 100unsigned 32-bit integer 101.It Li 64 102unsigned 64-bit integer 103.El 104.Pp 105For example, the function to atomically add two integers is called 106.Fn atomic_add_int . 107.Pp 108Certain architectures also provide operations for types smaller than 109.Dq Li int . 110.Pp 111.Bl -tag -offset indent -width short -compact 112.It Li char 113unsigned character 114.It Li short 115unsigned short integer 116.It Li 8 117unsigned 8-bit integer 118.It Li 16 119unsigned 16-bit integer 120.El 121.Pp 122These must not be used in machine-independent code, because the instructions 123to implement them efficiently may not be available. 124.Ss Memory Barriers 125Memory barriers are used to guarantee the order of data accesses in 126two ways. 127First, they specify hints to the compiler to not re-order or optimize the 128operations. 129Second, on architectures that do not guarantee ordered data accesses, 130special instructions or special variants of instructions are used to indicate 131to the processor that data accesses need to occur in a certain order. 132As a result, most of the atomic operations have three variants in order to 133include optional memory barriers. 134The first form just performs the operation without any explicit barriers. 135The second form uses a read memory barrier, and the third variant uses a write 136memory barrier. 137.Pp 138The second variant of each operation includes a read memory barrier. 139This barrier ensures that the effects of this operation are completed before the 140effects of any later data accesses. 141As a result, the operation is said to have acquire semantics as it acquires a 142pseudo-lock requiring further operations to wait until it has completed. 143To denote this, the suffix 144.Dq Li _acq 145is inserted into the function name immediately prior to the 146.Dq Li _ Ns Aq Fa type 147suffix. 148For example, to subtract two integers ensuring that any later writes will 149happen after the subtraction is performed, use 150.Fn atomic_subtract_acq_int . 151.Pp 152The third variant of each operation includes a write memory barrier. 153This ensures that all effects of all previous data accesses are completed 154before this operation takes place. 155As a result, the operation is said to have release semantics as it releases 156any pending data accesses to be completed before its operation is performed. 157To denote this, the suffix 158.Dq Li _rel 159is inserted into the function name immediately prior to the 160.Dq Li _ Ns Aq Fa type 161suffix. 162For example, to add two long integers ensuring that all previous 163writes will happen first, use 164.Fn atomic_add_rel_long . 165.Pp 166A practical example of using memory barriers is to ensure that data accesses 167that are protected by a lock are all performed while the lock is held. 168To achieve this, one would use a read barrier when acquiring the lock to 169guarantee that the lock is held before any protected operations are performed. 170Finally, one would use a write barrier when releasing the lock to ensure that 171all of the protected operations are completed before the lock is released. 172.Ss Multiple Processors 173The current set of atomic operations do not necessarily guarantee atomicity 174across multiple processors. 175To guarantee atomicity across processors, not only does the individual 176operation need to be atomic on the processor performing the operation, but 177the result of the operation needs to be pushed out to stable storage and the 178caches of all other processors on the system need to invalidate any cache 179lines that include the affected memory region. 180.Ss Semantics 181This section describes the semantics of each operation using a C like notation. 182.Bl -hang 183.It Fn atomic_add p v 184.Bd -literal -compact 185*p += v; 186.Ed 187.El 188.Pp 189The 190.Fn atomic_add 191functions are not implemented for the type 192.Dq Li cpumask . 193.Bl -hang 194.It Fn atomic_clear p v 195.Bd -literal -compact 196*p &= ~v; 197.Ed 198.It Fn atomic_cmpset dst old new 199.Bd -literal -compact 200if (*dst == old) { 201 *dst = new; 202 return 1; 203} else { 204 return 0; 205} 206.Ed 207.El 208.Pp 209The 210.Fn atomic_cmpset 211functions are not implemented for the types 212.Dq Li char , 213.Dq Li short , 214.Dq Li 8 , 215and 216.Dq Li 16 . 217.Bl -hang 218.It Fn atomic_fetchadd p v 219.Bd -literal -compact 220tmp = *p; 221*p += v; 222return tmp; 223.Ed 224.El 225.Pp 226The 227.Fn atomic_fetchadd 228functions are only implemented for the types 229.Dq Li int , 230.Dq Li long 231and 232.Dq Li 32 233and do not have any variants with memory barriers at this time. 234.Bl -hang 235.It Fn atomic_load addr 236.Bd -literal -compact 237return (*addr) 238.Ed 239.It Fn atomic_readandclear addr 240.Bd -literal -compact 241temp = *addr; 242*addr = 0; 243return (temp); 244.Ed 245.El 246.Pp 247The 248.Fn atomic_readandclear 249functions are not implemented for the types 250.Dq Li char , 251.Dq Li short , 252.Dq Li ptr , 253.Dq Li 8 , 254.Dq Li 16 , 255and 256.Dq Li cpumask 257and do 258not have any variants with memory barriers at this time. 259.Bl -hang 260.It Fn atomic_set p v 261.Bd -literal -compact 262*p |= v; 263.Ed 264.It Fn atomic_subtract p v 265.Bd -literal -compact 266*p -= v; 267.Ed 268.El 269.Pp 270The 271.Fn atomic_subtract 272functions are not implemented for the type 273.Dq Li cpumask . 274.Bl -hang 275.It Fn atomic_store p v 276.Bd -literal -compact 277*p = v; 278.Ed 279.El 280.Sh RETURN VALUES 281The 282.Fn atomic_cmpset 283function 284returns the result of the compare operation. 285The 286.Fn atomic_fetchadd , 287.Fn atomic_load , 288and 289.Fn atomic_readandclear 290functions 291return the value at the specified address. 292.\".Sh EXAMPLES 293.\"This example uses the 294.\".Fn atomic_cmpset_acq_ptr 295.\"and 296.\".Fn atomic_set_ptr 297.\"functions to obtain a sleep mutex and handle recursion. 298.\"Since the 299.\".Va mtx_lock 300.\"member of a 301.\".Vt "struct mtx" 302.\"is a pointer, the 303.\".Dq Li ptr 304.\"type is used. 305.\".Bd -literal 306.\"/* Try to obtain mtx_lock once. */ 307.\"#define _obtain_lock(mp, tid) \\ 308.\" atomic_cmpset_acq_ptr(&(mp)->mtx_lock, MTX_UNOWNED, (tid)) 309.\" 310.\"/* Get a sleep lock, deal with recursion inline. */ 311.\"#define _get_sleep_lock(mp, tid, opts, file, line) do { \\ 312.\" uintptr_t _tid = (uintptr_t)(tid); \\ 313.\" \\ 314.\" if (!_obtain_lock(mp, tid)) { \\ 315.\" if (((mp)->mtx_lock & MTX_FLAGMASK) != _tid) \\ 316.\" _mtx_lock_sleep((mp), _tid, (opts), (file), (line));\\ 317.\" else { \\ 318.\" atomic_set_ptr(&(mp)->mtx_lock, MTX_RECURSE); \\ 319.\" (mp)->mtx_recurse++; \\ 320.\" } \\ 321.\" } \\ 322.\"} while (0) 323.\".Ed 324.Sh HISTORY 325The 326.Fn atomic_add , 327.Fn atomic_clear , 328.Fn atomic_set , 329and 330.Fn atomic_subtract 331operations were first introduced in 332.Fx 3.0 . 333This first set only supported the types 334.Dq Li char , 335.Dq Li short , 336.Dq Li int , 337and 338.Dq Li long . 339.Pp 340The 341.Fn atomic_cmpset , 342.Fn atomic_load , 343.Fn atomic_readandclear , 344and 345.Fn atomic_store 346operations were added in 347.Fx 5.0 . 348The types 349.Dq Li 8 , 350.Dq Li 16 , 351.Dq Li 32 , 352.Dq Li 64 , 353and 354.Dq Li ptr 355and all of the acquire and release variants 356were added in 357.Fx 5.0 358as well. 359.Pp 360The 361.Fn atomic_fetchadd 362operations were added in 363.Fx 6.0 . 364.Pp 365The relaxed variants of 366.Fn atomic_load 367and 368.Fn atomic_store 369were added in 370.Fx 12.0 371and 372.Dx 6.5 . 373