admin-guide/mm/concepts.rst

5 The memory management in Linux is a complex system that evolved over the
7 systems from MMU-less microcontrollers to supercomputers. The memory
16 Virtual Memory Primer
19 The physical memory in a computer system is a limited resource and
20 even for systems that support memory hotplug there is a hard limit on
21 the amount of memory that can be installed. The physical memory is not
27 All this makes dealing directly with physical memory quite complex and
28 to avoid this complexity a concept of virtual memory was developed.
30 The virtual memory abstracts the details of physical memory from the
31 application software, allows to keep only needed information in the
32 physical memory (demand paging) and provides a mechanism for the
35 With virtual memory, each and every memory access uses a virtual
37 writes) from (or to) the system memory, it translates the `virtual`
38 address encoded in that instruction to a `physical` address that the
39 memory controller can understand.
41 The physical system memory is divided into page frames, or pages. The
47 Each physical memory page can be mapped as one or more virtual
50 memory address. The page tables are organized hierarchically.
55 levels. The pointer to the top level page table resides in a
58 virtual address are used to index an entry in the top level page
59 table. That entry is then used to access the next level in the
61 that level page table. The lowest bits in the virtual address define
67 The address translation requires several memory accesses and memory
72 large memory working set will experience performance hit because of
75 Many modern CPU architectures allow mapping of the memory pages
76 directly by the higher levels in the page table. For instance, on x86,
77 it is possible to map 2M and even 1G pages using entries in the second
78 and the third level page tables. In Linux such pages are called
80 improves TLB hit-rate and thus improves overall system performance.
82 There are two mechanisms in Linux that enable mapping of the physical
83 memory with the huge pages. The first one is `HugeTLB filesystem`, or
85 store. For the files created in this filesystem the data resides in
86 the memory and mapped using huge pages. The hugetlbfs is described at
87 Documentation/admin-guide/mm/hugetlbpage.rst.
92 the system memory should and can be mapped by the huge pages, THP
94 name. See Documentation/admin-guide/mm/transhuge.rst for more details
100 Often hardware poses restrictions on how different physical memory
101 ranges can be accessed. In some cases, devices cannot perform DMA to
102 all the addressable memory. In other cases, the size of the physical
103 memory exceeds the maximal addressable size of virtual memory and
104 special actions are required to access portions of the memory. Linux
105 groups memory pages into `zones` according to their possible
106 usage. For example, ZONE_DMA will contain memory that can be used by
107 devices for DMA, ZONE_HIGHMEM will contain memory that is not
111 The actual layout of the memory zones is hardware dependent as not all
118 Many multi-processor machines are NUMA - Non-Uniform Memory Access -
119 systems. In such systems the memory is arranged into banks that have
122 constructs an independent memory management subsystem. A node has its
124 counters. You can find more details about NUMA in
125 Documentation/mm/numa.rst` and in
126 Documentation/admin-guide/mm/numa_memory_policy.rst.
131 The physical memory is volatile and the common case for getting data
132 into the memory is to read it from files. Whenever a file is read, the
135 is placed in the page cache and eventually gets into the backing
140 Anonymous Memory
143 The `anonymous memory` or `anonymous mappings` represent memory that
146 call. Usually, the anonymous mappings only define virtual memory areas
148 in creation of a page table entry that references a special physical
160 memory allocated by user space processes etc.
163 memory management. The pages that can be freed at any time, either
167 reclaimable pages are page cache and anonymous memory.
169 In most cases, the pages holding internal kernel data and used as DMA
171 their user. Such pages are called `unreclaimable`. However, in certain
173 reclaimed. For instance, in-memory caches of filesystem metadata can
174 be re-read from the storage device and therefore it is possible to
175 discard them from the main memory when system is under memory
178 The process of freeing the reclaimable physical memory pages and
181 of the system. When the system is not loaded, most of the memory is free
186 asynchronously scan memory pages and either just free them if the data
188 device (remember those dirty pages?). As memory usage increases even
189 more and reaches another threshold - min watermark - an allocation
190 will trigger `direct reclaim`. In this case allocation is stalled
191 until enough memory pages are reclaimed to satisfy the request.
196 As the system runs, tasks allocate and free the memory and it becomes
197 fragmented. Although with virtual memory it is possible to present
199 necessary to allocate large physically contiguous memory areas. Such
201 buffer for DMA, or when THP allocates a huge page. Memory `compaction`
203 from the lower part of a memory zone to free pages in the upper part
208 Like reclaim, the compaction may happen asynchronously in the ``kcompactd``
209 daemon or synchronously as a result of a memory allocation request.
214 It is possible that on a loaded machine memory will be exhausted and the
215 kernel will be unable to reclaim enough memory to continue to operate. In
219 system health. The selected task is killed in a hope that after it exits
220 enough memory will be freed to continue normal operation.