Lines Matching full:memory
1 Memory Resource Controller
3 NOTE: The Memory Resource Controller has generically been referred to as the
4 memory controller in this document. Do not confuse memory controller
5 used here with the memory controller that is used in hardware.
9 When we mention a cgroup (cgroupfs's directory) with memory controller,
10 we call it "memory cgroup". When you see git-log and source code, you'll
14 Benefits and Purpose of the memory controller
16 The memory controller isolates the memory behaviour of a group of tasks
18 uses of the memory controller. The memory controller can be used to
21 Memory hungry applications can be isolated and limited to a smaller
22 amount of memory.
23 b. Create a cgroup with limited amount of memory, this can be used
25 c. Virtualization solutions can control the amount of memory they want
27 d. A CD/DVD burner could control the amount of memory used by the
29 of available memory.
39 - optionally, memory+swap usage can be accounted and limited.
47 Kernel memory support is work in progress, and the current version provides
55 memory.usage_in_bytes # show current res_counter usage for memory
57 memory.memsw.usage_in_bytes # show current res_counter usage for memory+Swap
59 memory.limit_in_bytes # set/show limit of memory usage
60 memory.memsw.limit_in_bytes # set/show limit of memory+Swap usage
61 memory.failcnt # show the number of memory usage hits limits
62 memory.memsw.failcnt # show the number of memory+Swap hits limits
63 memory.max_usage_in_bytes # show max memory usage recorded
64 memory.memsw.max_usage_in_bytes # show max memory+Swap usage recorded
65 memory.soft_limit_in_bytes # set/show soft limit of memory usage
66 memory.stat # show various statistics
67 memory.use_hierarchy # set/show hierarchical account enabled
68 memory.force_empty # trigger forced move charge to parent
69 memory.swappiness # set/show swappiness parameter of vmscan
71 memory.move_charge_at_immigrate # set/show controls of moving charges
72 memory.oom_control # set/show oom controls.
73 memory.numa_stat # show the number of memory usage per numa node
75 memory.kmem.tcp.limit_in_bytes # set/show hard limit for tcp buf memory
76 memory.kmem.tcp.usage_in_bytes # show current tcp buf memory allocation
80 The memory controller has a long history. A request for comments for the memory
82 there were several implementations for memory control. The goal of the
84 for memory control. The first RSS controller was posted by Balbir Singh[2]
88 to allow user space handling of OOM. The current memory controller is
92 2. Memory Control
94 Memory is a unique resource in the sense that it is present in a limited
97 memory, the same physical memory needs to be reused to accomplish the task.
99 The memory controller implementation has been divided into phases. These
102 1. Memory controller
104 3. Kernel user memory accounting and slab control
107 The memory controller is the first controller developed.
112 tracks the current memory usage and limit of the group of processes associated
113 with the controller. Each cgroup has a memory controller specific data
152 (*) page_cgroup structure is allocated at boot/memory-hotplug time.
186 the cgroup that brought it in -- this will happen on memory pressure).
190 be backed into memory in force, charges for pages are accounted against the
200 - memory.memsw.usage_in_bytes.
201 - memory.memsw.limit_in_bytes.
203 memsw means memory+swap. Usage of memory+swap is limited by
206 Example: Assume a system with 4G of swap. A task which allocates 6G of memory
207 (by mistake) under 2G memory limitation will use all swap.
212 * why 'memory+swap' rather than swap.
214 to move account from memory to swap...there is no change in usage of
215 memory+swap. In other words, when we want to limit the usage of swap without
216 affecting global LRU, memory+swap limit is better than just limiting swap from
219 * What happens when a cgroup hits memory.memsw.limit_in_bytes
220 When a cgroup hits memory.memsw.limit_in_bytes, it's useless to do swap-out
222 caches are dropped. But as mentioned above, global LRU can do swapout memory
223 from it for sanity of the system's memory management state. You can't forbid
230 to reclaim memory from the cgroup so as to make space for the new
261 2.7 Kernel Memory Extension (CONFIG_CGROUP_MEM_RES_CTLR_KMEM)
263 With the Kernel memory extension, the Memory Controller is able to limit
264 the amount of kernel memory used by the system. Kernel memory is fundamentally
265 different than user memory, since it can't be swapped out, which makes it
268 Kernel memory limits are not imposed for the root cgroup. Usage for the root
271 Currently no soft limit is implemented for kernel memory. It is future work
274 2.7.1 Current Kernel Memory resources accounted
276 * sockets memory pressure: some sockets protocols have memory pressure
277 thresholds. The Memory Controller allows them to be controlled individually
280 * tcp memory pressure: sockets memory pressure for the tcp protocol.
293 # mkdir /sys/fs/cgroup/memory
294 # mount -t cgroup none /sys/fs/cgroup/memory -o memory
297 # mkdir /sys/fs/cgroup/memory/0
298 # echo $$ > /sys/fs/cgroup/memory/0/tasks
300 Since now we're in the 0 cgroup, we can alter the memory limit:
301 # echo 4M > /sys/fs/cgroup/memory/0/memory.limit_in_bytes
309 # cat /sys/fs/cgroup/memory/0/memory.limit_in_bytes
313 # cat /sys/fs/cgroup/memory/0/memory.usage_in_bytes
319 availability of memory on the system. The user is required to re-read
322 # echo 1 > memory.limit_in_bytes
323 # cat memory.limit_in_bytes
326 The memory.failcnt field gives the number of times that the cgroup limit was
329 The memory.stat file gives accounting information. Now, the number of
336 Performance test is also important. To see pure memory controller's overhead,
345 Trying usual test under memory controller is always helpful.
353 2. The user is using anonymous memory and swap is turned off or too low
390 memory.force_empty interface is provided to make cgroup's memory usage empty.
394 # echo 0 > memory.force_empty
396 Almost all pages tracked by this memory cgroup will be unmapped and freed.
407 memory.stat file includes following statistics
409 # per-memory cgroup local status
410 cache - # of bytes of page cache memory.
411 rss - # of bytes of anonymous and swap cache memory.
413 pgpgin - # of charging events to the memory cgroup. The charging
416 pgpgout - # of uncharging events to the memory cgroup. The uncharging
419 inactive_anon - # of bytes of anonymous memory and swap cache memory on
421 active_anon - # of bytes of anonymous and swap cache memory on active
423 inactive_file - # of bytes of file-backed memory on inactive LRU list.
424 active_file - # of bytes of file-backed memory on active LRU list.
425 unevictable - # of bytes of memory that cannot be reclaimed (mlocked etc).
427 # status considering hierarchy (see memory.use_hierarchy settings)
429 hierarchical_memory_limit - # of bytes of memory limit with regard to hierarchy
430 under which the memory cgroup is
431 hierarchical_memsw_limit - # of bytes of memory+swap limit with regard to
432 hierarchy under which memory cgroup is.
459 Only anonymous and swap cache memory is listed as part of 'rss' stat.
461 amount of physical memory used by the cgroup.
464 file_mapped is accounted only when the memory cgroup is owner of page
478 A memory cgroup provides memory.failcnt and memory.memsw.failcnt files.
480 hit its limit. When a memory cgroup hits a limit, failcnt increases and
481 memory under it will be reclaimed.
484 # echo 0 > .../memory.failcnt
488 For efficiency, as other kernel components, memory cgroup uses some optimization
490 method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz
492 If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP)
493 value in memory.stat(see 5.2).
504 each memcg. The ouput format of memory.numa_stat is:
515 The memory controller supports a deep hierarchy and hierarchical accounting.
528 In the diagram above, with hierarchical accounting enabled, all memory
530 that has memory.use_hierarchy enabled. If one of the ancestors goes over its
536 A memory cgroup by default disables the hierarchy feature. Support
537 can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup
539 # echo 1 > memory.use_hierarchy
543 # echo 0 > memory.use_hierarchy
554 Soft limits allow for greater sharing of memory. The idea behind soft limits
555 is to allow control groups to use as much of the memory as needed, provided
557 a. There is no memory contention
560 When the system detects memory contention or low memory, control groups
563 sure that one control group does not starve the others of memory.
566 no guarantees, but it does its best to make sure that when memory is
567 heavily contended for, memory is allocated based on the soft limit
576 # echo 256M > memory.soft_limit_in_bytes
580 # echo 1G > memory.soft_limit_in_bytes
583 reclaiming memory for balancing between memory cgroups
597 writing to memory.move_charge_at_immigrate of the destination cgroup.
601 # echo (some positive value) > memory.move_charge_at_immigrate
608 try to make space by reclaiming memory. Task migration may fail if we
614 # echo 0 > memory.move_charge_at_immigrate
621 memory cgroup.
629 1 | A charge of file pages(normal file, tmpfs file(e.g. ipc shared memory)
645 9. Memory thresholds
647 Memory cgroup implements memory thresholds using cgroups notification
648 API (see cgroups.txt). It allows to register multiple memory and memsw
653 - open memory.usage_in_bytes or memory.memsw.usage_in_bytes;
654 - write string like "<event_fd> <fd of memory.usage_in_bytes> <threshold>" to
657 Application will be notified through eventfd when memory usage crosses
664 memory.oom_control file is for OOM notification and other controls.
666 Memory cgroup implements OOM notifier using cgroup notification
672 - open memory.oom_control file
673 - write string like "<event_fd> <fd of memory.oom_control>" to
679 You can disable OOM-killer by writing "1" to memory.oom_control file, as:
681 #echo 1 > memory.oom_control
685 in memory cgroup's OOM-waitqueue when they request accountable memory.
687 For running them, you have to relax the memory cgroup's OOM status by
698 under_oom 0 or 1 (if 1, the memory cgroup is under OOM, tasks may
711 Overall, the memory controller has been a stable controller and has been
716 1. Singh, Balbir. RFC: Memory Controller, http://lwn.net/Articles/206697/
717 2. Singh, Balbir. Memory Controller (RSS Control),
732 10. Singh, Balbir. Memory controller v6 test results,
734 11. Singh, Balbir. Memory controller introduction (v6),
736 12. Corbet, Jonathan, Controlling memory use in cgroups,