admin-guide/mm/numa_memory_policy.rst

4 NUMA Memory Policy
7 What is NUMA Memory Policy?
10 In the Linux kernel, "memory policy" determines from which node the kernel will
11 allocate memory in a NUMA system or in an emulated NUMA system.  Linux has
12 supported platforms with Non-Uniform Memory Access architectures since 2.4.?.
13 The current memory policy support was added to Linux 2.6 around May 2004.  This
14 document attempts to describe the concepts and APIs of the 2.6 memory policy
17 Memory policies should not be confused with cpusets
18 (``Documentation/admin-guide/cgroup-v1/cpusets.rst``)
20 memory may be allocated by a set of processes. Memory policies are a
21 programming interface that a NUMA-aware application can take advantage of.  When
22 both cpusets and policies are applied to a task, the restrictions of the cpuset
23 takes priority.  See :ref:`Memory Policies and cpusets <mem_pol_and_cpusets>`
26 Memory Policy Concepts
29 Scope of Memory Policies
30 ------------------------
32 The Linux kernel supports _scopes_ of memory policy, described here from
33 most general to most specific:
41 	up, the system default policy will be set to interleave
42 	allocations across all nodes with "sufficient" memory, so as
43 	not to overload the initial boot node with boot-time
47 	this is an optional, per-task policy.  When defined for a
52 	task policy "fall back" to the System Default Policy.
54 	The task policy applies to the entire address space of a task. Thus,
57 	to establish the task policy for a child task exec()'d from an
58 	executable image that has no awareness of memory policy.  See the
59 	:ref:`Memory Policy APIs <memory_policy_apis>` section,
61 	that a task may use to set/change its task/process policy.
63 	In a multi-threaded task, task policies apply only to the thread
69 	A task policy applies only to pages allocated after the policy is
77 	A "VMA" or "Virtual Memory Area" refers to a range of a task's
80 	:ref:`Memory Policy APIs <memory_policy_apis>` section,
81 	below, for an overview of the mbind() system call used to set a VMA
87 	back to the task policy, which may itself fall back to the
92 	* VMA policy applies ONLY to anonymous pages.  These include
96 	  applied to a file mapping, it will be ignored if the mapping
99 	  an anonymous page is allocated on an attempt to write to the
100 	  mapping-- i.e., at Copy-On-Write.
103 	  virtual address space--a.k.a. threads--independent of when
105 	  fork().  However, because VMA policies refer to a specific
108 	  are NOT inheritable across exec().  Thus, only NUMA-aware
111 	* A task may install a new VMA policy on a sub-range of a
113 	  the existing virtual memory area into 2 or 3 VMAs, each with
116 	* By default, VMA policy applies only to pages allocated after
121 	  call, so that page contents can be moved to match a newly
125 	Conceptually, shared policies apply to "memory objects" mapped
128 	policies--using the mbind() system call specifying a range of
130 	VMA policies, which can be considered to be an attribute of a
132 	directly to the shared object.  Thus, all tasks that attach to
136 	As of 2.6.22, only shared memory segments, created by shmget() or
138 	policy support was added to Linux, the associated data structures were
139 	added to hugetlbfs shmem segments.  At the time, hugetlbfs did not
140 	support allocation at fault time--a.k.a lazy allocation--so hugetlbfs
141 	shmem segments were never "hooked up" to the shared policy support.
156 	Thus, different tasks that attach to a shared memory segment can have
159 	a shared memory region, when one task has installed shared policy on
162 Components of Memory Policies
163 -----------------------------
165 A NUMA memory policy consists of a "mode", optional mode flags, and
168 and the optional set of nodes can be viewed as the arguments to the
171 Internally, memory policies are implemented by a reference counted
173 discussed in context, below, as required to explain the behavior.
175 NUMA memory policy supports the following 4 behavioral modes:
177 Default Mode--MPOL_DEFAULT
178 	This mode is only used in the memory policy APIs.  Internally,
179 	MPOL_DEFAULT is converted to the NULL memory policy in all
180 	policy scopes.  Any existing non-default policy will simply be
182 	MPOL_DEFAULT means "fall back to the next most specific policy
185 	For example, a NULL or default task policy will fall back to the
187 	back to the task policy.
189 	When specified in one of the memory policy APIs, the Default mode
192 	It is an error for the set of nodes specified for this policy to
193 	be non-empty.
196 	This mode specifies that memory must come from the set of
197 	nodes specified by the policy.  Memory will be allocated from
198 	the node in the set with sufficient free memory that is
199 	closest to the node where the allocation takes place.
208 	Internally, the Preferred policy uses a single node--the
216 	It is possible for the user to specify that local allocation
228 	For allocation of anonymous pages and shared memory pages,
232 	nodes specified by the policy.  It then attempts to allocate a
240 	maintained per task.  This counter wraps around to the lowest
242 	This will tend to spread the pages out over the nodes
248 NUMA memory policy supports the following optional mode flags:
253 	nodes changes after the memory policy has been defined.
257 	nodemask (Bind, Interleave) is remapped to the new set of
261 	With this flag, if the user-specified nodes overlap with the
262 	nodes allowed by the task's cpuset, then the memory policy is
263 	applied to their intersection.  If the two sets of nodes do not
266 	For example, consider a task that is attached to a cpuset with
267 	mems 1-3 that sets an Interleave policy over the same set.  If
268 	the cpuset's mems change to 3-5, the Interleave will now occur
281 	by the user will be mapped relative to the set of the task or VMA's
282 	set of allowed nodes.  The kernel stores the user-passed nodemask,
284 	be remapped relative to the new set of allowed nodes.
289 	remapped to the new set of allowed nodes.  That remap may not
290 	preserve the relative nature of the user's passed nodemask to its
292 	1,3,5 may be remapped to 7-9 and then to 1-3 if the set of
293 	allowed nodes is restored to its original state.
296 	the user's passed nodemask are relative to the set of allowed
301 	relative to task or VMA's set of allowed nodes.
305 	the user's nodemask when the set of allowed nodes is only 0-3),
306 	then the remap wraps around to the beginning of the nodemask and,
309 	For example, consider a task that is attached to a cpuset with
310 	mems 2-5 that sets an Interleave policy over the same set with
311 	MPOL_F_RELATIVE_NODES.  If the cpuset's mems change to 3-7, the
312 	interleave now occurs over nodes 3,5-7.  If the cpuset's mems
313 	then change to 0,2-3,5, then the interleave occurs over nodes
314 	0,2-3,5.
316 	Thanks to the consistent remapping, applications preparing
317 	nodemasks to specify memory policies using this flag should
318 	disregard their current, actual cpuset imposed memory placement
320 	memory nodes 0 to N-1, where N is the number of memory nodes the
321 	policy is intended to manage.  Let the kernel then remap to the
322 	set of memory nodes allowed by the task's cpuset, as that may
330 Memory Policy Reference Counting
333 To resolve use/free races, struct mempolicy contains an atomic reference
336 the structure back to the mempolicy kmem cache when the reference count
337 goes to zero.
339 When a new memory policy is allocated, its reference count is initialized
340 to '1', representing the reference held by the task that is installing the
341 new policy.  When a pointer to a memory policy structure is stored in another
345 During run-time "usage" of the policy, we attempt to minimize atomic operations
346 on the reference count, as this can lead to cache lines bouncing between cpus
353 2) examination of the policy to determine the policy mode and associated node
357    BIND policy nodemask is used, by reference, to filter ineligible nodes.
362 1) we never need to get/free the system default policy as this is never
365 2) for querying the policy, we do not need to take an extra reference on the
379    shared memory policy while another task, with a distinct mmap_lock, is
380    querying or allocating a page based on the policy.  To resolve this
382    to the shared policy during lookup while holding a spin lock on the shared
386    used for non-shared policies.  For this reason, shared policies are marked
387    as such, and the extra reference is dropped "conditionally"--i.e., only
392    more expensive to use in the page allocation path.  This is especially
393    true for shared policies on shared memory regions shared by tasks running
395    falling back to task or system default policy for shared memory regions,
396    or by prefaulting the entire shared memory region into memory and locking
401 Memory Policy APIs
404 Linux supports 3 system calls for controlling memory policy.  These APIS
415 Set [Task] Memory Policy::
420 Set's the calling task's "task/process memory policy" to mode
422 'nmask'.  'nmask' points to a bit mask of node ids containing at least
430 Get [Task] Memory Policy or Related Information::
436 Queries the "task/process memory policy" of the calling task, or the
456 Memory Policy Command Line Interface
459 Although not strictly part of the Linux implementation of memory policy,
460 a command line tool, numactl(8), exists that allows one to:
465 + set the shared policy for a shared memory segment via mbind(2)
467 The numactl(8) tool is packaged with the run-time version of the library
468 containing the memory policy system call wrappers.  Some distributions
469 package the headers and compile-time libraries in a separate development
474 Memory Policies and cpusets
477 Memory policies work within cpusets as described above.  For memory policies
478 that require a node or set of nodes, the nodes are restricted to the set of
482 specified for the policy and the set of nodes with memory is used.  If the
487 The interaction of memory policies and cpusets can be problematic when tasks
488 in two cpusets share access to a memory region, such as shared memory segments
492 this information requires "stepping outside" the memory policy APIs to use the
494 be attaching to the shared region.  Furthermore, if the cpusets' allowed
495 memory sets are disjoint, "local" allocation is the only valid policy.