Documentation/core-api/this_cpu_ops.rst

8 this_cpu operations are a way of optimizing access to per cpu
11 the cpu permanently stored the beginning of the per cpu	area for a
14 this_cpu operations add a per cpu variable offset to the processor
15 specific per cpu base and encode that operation in the instruction
16 operating on the per cpu variable.
24 Read-modify-write operations are of particular interest. Frequently
32 synchronization is not necessary since we are dealing with per cpu
37 Please note that accesses by remote processors to a per cpu area are
65 ------------------------------------
68 per cpu area. It is then possible to simply use the segment override
69 to relocate a per cpu relative address to the proper per cpu area for
70 the processor. So the relocation to the per cpu base is encoded in the
85 from that address which occurs with the per cpu operations. Before
102 	int cpu;
104 	cpu = get_cpu();
105 	y = per_cpu_ptr(&x, cpu);
109 Note that these operations can only be used on per cpu data that is
112 per cpu counters is correctly incremented. However, there is no
116 meaningless. The sum of all the per cpu counters is the only value
119 Per cpu variables are used for performance reasons. Bouncing cache
121 the same code paths.  Since each processor has its own per cpu
123 has to be paid for this optimization is the need to add up the per cpu
128 ------------------
134 Takes the offset of a per cpu variable (&x !) and returns the address
135 of the per cpu variable that belongs to the currently executing
138 available. Instead, the offset of the local per cpu area is simply
139 added to the per cpu offset.
143 disabled. The pointer is then used to access local per cpu data in a
144 critical section. When preemption is re-enabled this pointer is usually
145 no longer useful since it may no longer point to per cpu data of the
148 The special cases where it makes sense to obtain a per-CPU pointer in
150 to handle cases where two different CPUs are accessing the same per cpu
151 variable, which might well be that of a third CPU.  These use cases are
153 of counters as a pair of per-CPU variables, and rcu_read_lock_nmisafe()
154 uses raw_cpu_ptr() to get a pointer to some CPU's counter, and uses
158 Per cpu variables and offsets
159 -----------------------------
161 Per cpu variables have *offsets* to the beginning of the per cpu
164 added to a base pointer of a per cpu area of a processor in order to
167 Therefore the use of x or &x outside of the context of per cpu
175 In the context of per cpu operations the above implies that x is a per
176 cpu variable. Most this_cpu operations take a cpu variable.
182 &x and hence p is the *offset* of a per cpu variable. this_cpu_ptr()
183 takes the offset of a per cpu variable which makes this look a bit
187 Operations on a field of a per cpu structure
188 --------------------------------------------
210 	this_cpu_dec(ps->m);
212 	z = this_cpu_inc_return(ps->n);
222 	pp->m--;
224 	z = pp->n++;
228 ------------------------
231 these per cpu local operations. In that case the operation must be
233 that are guaranteed to be atomic and then re-enable interrupts. Doing
240 preemption. If a per cpu variable is not used in an interrupt context
262 Will increment x and will not fall-back to code that disables
264 address relocation and a Read-Modify-Write operation in the same
268 &this_cpu_ptr(pp)->n vs this_cpu_ptr(&pp->n)
269 --------------------------------------------
281 Remote access to per cpu data
282 ------------------------------
284 Per cpu data structures are designed to be used by one cpu exclusively.
286 be "atomic" as no other CPU has access to these data structures.
288 There are special cases where you might need to access per cpu data
297 the remote CPU and perform the update to its per cpu area.
299 To access per-cpu data structure remotely, typically the per_cpu_ptr()
305 	struct data *p = per_cpu_ptr(&datap, cpu);
318 per cpu data. Write accesses can cause unique problems due to the
322 the following scenario that occurs because two per cpu variables
323 share a cache-line but the relaxed synchronization is applied to
324 only one process updating the cache-line.
341 of a remote write to the per cpu area of another processor.
346 missing local cache line of a per cpu area, its performance and hence