Documentation/core-api/this_cpu_ops.rst

8 this_cpu operations are a way of optimizing access to per cpu
9 variables associated with the *currently* executing processor. This is
11 the cpu permanently stored the beginning of the per cpu	area for a
12 specific processor).
14 this_cpu operations add a per cpu variable offset to the processor
15 specific per cpu base and encode that operation in the instruction
16 operating on the per cpu variable.
21 processor is not changed between the calculation of the address and
24 Read-modify-write operations are of particular interest. Frequently
32 synchronization is not necessary since we are dealing with per cpu
33 data specific to the currently executing processor. Only the current
34 processor should be accessing that variable and therefore there are no
37 Please note that accesses by remote processors to a per cpu area are
66 ------------------------------------
69 per cpu area. It is then possible to simply use the segment override
70 to relocate a per cpu relative address to the proper per cpu area for
71 the processor. So the relocation to the per cpu base is encoded in the
86 from that address which occurs with the per cpu operations. Before
88 prevent the kernel from moving the thread to a different processor
110 Note that these operations can only be used on per cpu data that is
111 reserved for a specific processor. Without disabling preemption in the
113 per cpu counters is correctly incremented. However, there is no
116 the value of the individual counters for each processor are
117 meaningless. The sum of all the per cpu counters is the only value
120 Per cpu variables are used for performance reasons. Bouncing cache
122 the same code paths.  Since each processor has its own per cpu
124 has to be paid for this optimization is the need to add up the per cpu
129 ------------------
135 Takes the offset of a per cpu variable (&x !) and returns the address
136 of the per cpu variable that belongs to the currently executing
137 processor.  this_cpu_ptr avoids multiple steps that the common
138 get_cpu/put_cpu sequence requires. No processor number is
139 available. Instead, the offset of the local per cpu area is simply
140 added to the per cpu offset.
144 access local per cpu data in a critical section. When preemption
145 is re-enabled this pointer is usually no longer useful since it may
146 no longer point to per cpu data of the current processor.
149 Per cpu variables and offsets
150 -----------------------------
152 Per cpu variables have *offsets* to the beginning of the per cpu
155 added to a base pointer of a per cpu area of a processor in order to
158 Therefore the use of x or &x outside of the context of per cpu
166 In the context of per cpu operations the above implies that x is a per
173 &x and hence p is the *offset* of a per cpu variable. this_cpu_ptr()
174 takes the offset of a per cpu variable which makes this look a bit
178 Operations on a field of a per cpu structure
179 --------------------------------------------
201 	this_cpu_dec(ps->m);
203 	z = this_cpu_inc_return(ps->n);
213 	pp->m--;
215 	z = pp->n++;
219 ------------------------
222 these per cpu local operations. In that case the operation must be
224 that are guaranteed to be atomic and then re-enable interrupts. Doing
226 change the processor we are executing on then there is no reason to
231 preemption. If a per cpu variable is not used in an interrupt context
254 Will increment x and will not fall-back to code that disables
256 address relocation and a Read-Modify-Write operation in the same
260 &this_cpu_ptr(pp)->n vs this_cpu_ptr(&pp->n)
261 --------------------------------------------
273 Remote access to per cpu data
274 ------------------------------
276 Per cpu data structures are designed to be used by one cpu exclusively.
280 There are special cases where you might need to access per cpu data
289 the remote CPU and perform the update to its per cpu area.
291 To access per-cpu data structure remotely, typically the per_cpu_ptr()
310 per cpu data. Write accesses can cause unique problems due to the
314 the following scenario that occurs because two per cpu variables
315 share a cache-line but the relaxed synchronization is applied to
316 only one process updating the cache-line.
329 remotely from one processor and the local processor would use this_cpu ops
333 of a remote write to the per cpu area of another processor.
336 mind that a remote write will evict the cache line from the processor
337 that most likely will access it. If the processor wakes up and finds a
338 missing local cache line of a per cpu area, its performance and hence