admin-guide/pm/cpuidle.rst

1 .. SPDX-License-Identifier: GPL-2.0
5 .. |cpufreq| replace:: :doc:`CPU Performance Scaling <cpufreq>`
8 CPU Idle Time Management
19 Modern processors are generally able to enter states in which the execution of
21 memory or executed.  Those states are the *idle* states of the processor.
23 Since part of the processor hardware is not used in idle states, entering them
27 CPU idle time management is an energy-efficiency feature concerned about using
28 the idle states of processors for this purpose.
31 ------------
33 CPU idle time management operates on CPUs as seen by the *CPU scheduler* (that
37 software as individual single-core processors.  In other words, a CPU is an
43 program) at a time, it is a CPU.  In that case, if the hardware is asked to
44 enter an idle state, that applies to the processor as a whole.
46 Second, if the processor is multi-core, each core in it is able to follow at
52 enter an idle state, that applies to the core that asked for it in the first
56 except for one have been put into idle states at the "core level" and the
57 remaining core asks the processor to enter an idle state, that may trigger it
58 to put the whole larger unit into an idle state which also will affect the
61 Finally, each core in a multi-core processor may be able to follow more than one
66 multiple individual single-core "processors", referred to as *hardware threads*
67 (or hyper-threads specifically on Intel hardware), that each can follow one
68 sequence of instructions.  Then, the hardware threads are CPUs from the CPU idle
69 time management perspective and if the processor is asked to enter an idle state
70 by one of them, the hardware thread (or CPU) that asked for it is stopped, but
72 core also have asked the processor to enter an idle state.  In that situation,
73 the core may be put into an idle state individually or a larger unit containing
74 it may be put into an idle state as a whole (if the other cores within the
75 larger unit are in idle states already).
77 Idle CPUs
78 ---------
81 *idle* by the Linux kernel when there are no tasks to run on them except for the
82 special "idle" task.
84 Tasks are the CPU scheduler's representation of work.  Each task consists of a
87 processor every time the task's code is run by a CPU.  The CPU scheduler
90 Tasks can be in various states.  In particular, they are *runnable* if there are
91 no specific conditions preventing their code from being run by a CPU as long as
92 there is a CPU available for that (for example, they are not waiting for any
93 events to occur or similar).  When a task becomes runnable, the CPU scheduler
95 tasks assigned to it, the CPU will load the given task's context and run its
97 another CPU).  [If there are multiple runnable tasks assigned to one CPU
101 The special "idle" task becomes runnable if there are no other runnable tasks
102 assigned to the given CPU and the CPU is then regarded as idle.  In other words,
103 in Linux idle CPUs run the code of the "idle" task called *the idle loop*.  That
104 code may cause the processor to be put into one of its idle states, if they are
106 idle states, or there is not enough time to spend in an idle state before the
108 available idle states from being used, the CPU will simply execute more or less
112 .. _idle-loop:
114 The Idle Loop
117 The idle loop code takes two major steps in every iteration of it.  First, it
118 calls into a code module referred to as the *governor* that belongs to the CPU
119 idle time management subsystem called ``CPUIdle`` to select an idle state for
120 the CPU to ask the hardware to enter.  Second, it invokes another code module
122 processor hardware to enter the idle state selected by the governor.
124 The role of the governor is to find an idle state most suitable for the
125 conditions at hand.  For this purpose, idle states that the hardware can be
127 the platform or the processor architecture and organized in a one-dimensional
133 Each idle state present in that array is characterized by two parameters to be
134 taken into account by the governor, the *target residency* and the (worst-case)
138 the shallower idle states instead.  [The "depth" of an idle state roughly
140 latency, in turn, is the maximum time it will take a CPU asking the processor
141 hardware to enter an idle state to start executing the first instruction after a
151 CPU depends on can spend in an idle state, including the time necessary to enter
152 and exit it.  However, the CPU may be woken up by a non-timer event at any time
154 when that may happen.  The governor can only see how much time the CPU actually
155 was idle after it has been woken up (that time will be referred to as the *idle
157 time until the closest timer to estimate the idle duration in future.  How the
162 There are four ``CPUIdle`` governors available, ``menu``, `TEO <teo-gov_>`_,
165 tick can be `stopped by the idle loop <idle-cpus-and-tick_>`_.  Available
169 :file:`current_governor` file under :file:`/sys/devices/system/cpu/cpuidle/`
176 hardcoded idle states information and the other able to read that information
183 file under :file:`/sys/devices/system/cpu/cpuidle/` in ``sysfs``.
186 .. _idle-cpus-and-tick:
188 Idle CPUs and The Scheduler Tick
192 the time sharing strategy of the CPU scheduler.  Of course, if there are
193 multiple runnable tasks assigned to one CPU at the same time, the only way to
195 share the available CPU time.  Namely, in rough approximation, each task is
196 given a slice of the CPU time to run its code, subject to the scheduling class,
197 prioritization and so on and when that time slice is used up, the CPU should be
199 may not want to give the CPU away voluntarily, however, and the scheduler tick
203 The scheduler tick is problematic from the CPU idle time management perspective,
206 Thus, if the tick is allowed to trigger on idle CPUs, it will not make sense
207 for them to ask the hardware to enter idle states with target residencies above
208 the tick period length.  Moreover, in that case the idle duration of any CPU
210 exiting idle states due to the tick wakeups on idle CPUs will be wasted.
212 Fortunately, it is not really necessary to allow the tick to trigger on idle
214 "idle" one.  In other words, from the CPU scheduler perspective, the only user
215 of the CPU time on them is the idle loop.  Since the time of an idle CPU need
217 tick goes away if the given CPU is idle.  Consequently, it is possible to stop
218 the scheduler tick entirely on idle CPUs in principle, even though that may not
221 Whether or not it makes sense to stop the scheduler tick in the idle loop
223 (non-tick) timer due to trigger within the tick range, stopping the tick clearly
225 reprogrammed in that case.  Second, if the governor is expecting a non-timer
227 be harmful.  Namely, in that case the governor will select an idle state with
229 going to be relatively shallow.  The governor really cannot select a deep idle
235 in the shallow idle state selected by the governor, which will be a waste of
238 governor will select a relatively deep idle state, so the tick should be stopped
239 so that it does not wake up the CPU too early.
246 The kernel can be configured to disable stopping the scheduler tick in the idle
247 loop altogether.  That can be done through the build-time configuration of it
251 ignored by the idle loop code and the tick is never stopped.
254 stopped on idle CPUs are referred to as *tickless* systems and they are
255 generally regarded as more energy-efficient than the systems running kernels in
261 .. _menu-gov:
268 Namely, when invoked to select an idle state for a CPU (i.e. an idle state that
269 the CPU will ask the processor hardware to enter), it attempts to predict the
270 idle duration and uses the predicted value for idle state selection.
273 idle duration prediction.  Namely, it saves the last 8 observed idle duration
274 values and, when predicting the idle duration next time, it computes the average
279 which one is farther from the average) of the saved observed idle duration
286 sufficiently large, the next idle duration is not likely to be above the largest
287 idle duration value still in that set, so that value is taken as the predicted
288 next idle duration.  Finally, if the set of data points still under
291 If the preliminary prediction of the next idle duration computed this way is
295 next CPU wakeup.  It is used to determine the sleep length range, which in turn
304 selecting the idle state for the CPU) is updated after the CPU has been woken
305 up and the closer the sleep length is to the observed idle duration, the closer
308 falls into to obtain an approximation of the predicted idle duration that is
310 the two is taken as the final idle duration prediction.
312 If the "typical interval" value is small, which means that the CPU is likely
314 be costly and the idle duration is simply predicted to equal the "typical
317 Now, the governor is ready to walk the list of idle states and choose one of
319 the predicted idle duration and the exit latency of it with the with the latency
320 limit coming from the power management quality of service, or `PM QoS <cpu-pm-qos_>`_,
322 idle duration, but still below it, and exit latency that does not exceed the
325 In the final step the governor may still need to refine the idle state selection
326 if it has not decided to `stop the scheduler tick <idle-cpus-and-tick_>`_.  That
327 happens if the idle duration predicted by it is less than the tick period and
328 the tick has not been stopped already (in a previous iteration of the idle
335 .. _teo-gov:
342 <menu-gov_>`_: it always tries to find the deepest idle state suitable for the
345 .. kernel-doc:: drivers/cpuidle/governors/teo.c
346    :doc: teo-description
348 .. _idle-states-representation:
350 Representation of Idle States
353 For the CPU idle time management purposes all of the physical idle states
354 supported by the processor have to be represented as a one-dimensional array of
355 |struct cpuidle_state| objects each allowing an individual (logical) CPU to ask
356 the processor hardware to enter an idle state of certain properties.  If there
358 cover a combination of idle states supported by the units at different levels of
360 of it <idle-loop_>`_, must reflect the properties of the idle state at the
361 deepest level (i.e. the idle state of the unit containing all of the other
365 a "module" and suppose that asking the hardware to enter a specific idle state
367 enter a specific idle state of its own (say "MX") if the other core is in idle
368 state "X" already.  In other words, asking for idle state "X" at the "core"
369 level gives the hardware a license to go as deep as to idle state "MX" at the
371 asking for idle state "X" may just end up in that state by itself instead).
373 idle state "X" must reflect the minimum time to spend in idle state "MX" of
375 time the CPU needs to be idle to save any energy in case the hardware enters
377 the exit time of idle state "MX" of the module (and usually its entry time too),
378 because that is the maximum delay between a wakeup signal and the time the CPU
384 hierarchy of units inside them, however.  In those cases asking for an idle
387 handling of the hierarchy.  Then, the definition of the idle state objects is
388 entirely up to the driver, but still the physical properties of the idle state
390 used by the governor for idle state selection (for instance, the actual exit
391 latency of that idle state must not exceed the exit latency parameter of the
392 idle state object selected by the governor).
394 In addition to the target residency and exit latency idle state parameters
395 discussed above, the objects representing idle states each contain a few other
396 parameters describing the idle state and a pointer to the function to run in
400 statistics of the given idle state.  That information is exposed by the kernel
403 For each CPU in the system, there is a :file:`/sys/devices/system/cpu/cpu<N>/cpuidle/`
405 CPU at the initialization time.  That directory contains a set of subdirectories
406 called :file:`state0`, :file:`state1` and so on, up to the number of idle state
407 objects defined for the given CPU minus one.  Each of these directories
408 corresponds to one idle state object and the larger the number in its name, the
409 deeper the (effective) idle state represented by it.  Each of them contains
410 a number of files (attributes) representing the properties of the idle state
414 	Total number of times this idle state had been asked for, but the
415 	observed idle duration was certainly too short to match its target
419 	Total number of times this idle state had been asked for, but certainly
420 	a deeper idle state would have been a better match for the observed idle
424 	Description of the idle state.
427 	Whether or not this idle state is disabled.
433 	Exit latency of the idle state in microseconds.
436 	Name of the idle state.
439 	Power drawn by hardware in this idle state in milliwatts (if specified,
443 	Target residency of the idle state in microseconds.
446 	Total time spent in this idle state by the given CPU (as measured by the
450 	Total number of times the hardware has been asked by the given CPU to
451 	enter this idle state.
454 	Total number of times a request to enter this idle state on the given
455 	CPU was rejected.
463 given idle state is disabled for this particular CPU, which means that the
464 governor will never select it for this particular CPU and the ``CPUIdle``
465 driver will never ask the hardware to enter it for that CPU as a result.
466 However, disabling an idle state for one CPU does not prevent it from being
469 governor is implemented, disabling an idle state prevents that governor from
470 selecting any idle states deeper than the disabled one too.]
472 If the :file:`disable` attribute contains 0, the given idle state is enabled for
473 this particular CPU, but it still may be disabled for some or all of the other
474 CPUs in the system at the same time.  Writing 1 to it causes the idle state to
475 be disabled for this particular CPU and writing 0 to it allows the governor to
476 take it into consideration for the given CPU and the driver to ask for it,
480 The :file:`power` attribute is not defined very well, especially for idle state
481 objects representing combinations of idle states at different levels of the
482 hierarchy of units in the processor, and it generally is hard to obtain idle
488 really spent by the given CPU in the given idle state, because it is measured by
490 this idle state and entered a shallower one instead of it (or even it did not
491 enter any idle state at all).  The kernel can only measure the time span between
492 asking the hardware to enter an idle state and the subsequent wakeup of the CPU
494 Moreover, if the idle state object in question represents a combination of idle
495 states at different levels of the hierarchy of units in the processor,
498 much time has been spent by the hardware in different idle states supported by
499 it is to use idle state residency counters in the hardware, if available.
501 Generally, an interrupt received when trying to enter an idle state causes the
502 idle state entry request to be rejected, in which case the ``CPUIdle`` driver
504 and :file:`rejected` files report the number of times the given idle state
507 .. _cpu-pm-qos:
514 energy-efficiency features of the kernel to prevent performance from dropping
517 CPU idle time management can be affected by PM QoS in two ways, through the
518 global CPU latency limit and through the resume latency constraints for
523 signed 32-bit integer) to it.  In turn, the resume latency constraint for a CPU
525 32-bit integer) to the :file:`power/pm_qos_resume_latency_us` file under
526 :file:`/sys/devices/system/cpu/cpu<N>/` in ``sysfs``, where the CPU number
535 global CPU latency limit and for each individual CPU, aggregates them and
540 PM QoS request to be created and added to a global priority list of CPU latency
546 that effective value will be set as a new CPU latency limit.  Thus requesting a
558 with that file descriptor to be removed from the global priority list of CPU
563 In turn, for each CPU there is one resume latency PM QoS request associated with
565 :file:`/sys/devices/system/cpu/cpu<N>/` in ``sysfs`` and writing to it causes
570 practice is to pin a process to the CPU in question and let it use the
574 CPU in question every time the list of requests is updated this way or another
577 CPU idle time governors are expected to regard the minimum of the global
578 (effective) CPU latency limit and the effective resume latency constraint for
579 the given CPU as the upper limit for the exit latency of the idle states that
580 they are allowed to select for that CPU.  They should never select any idle
581 states with exit latency beyond that limit.
584 Idle States Control Via Kernel Command Line
587 In addition to the ``sysfs`` interface allowing individual idle states to be
588 `disabled for individual CPUs <idle-states-representation_>`_, there are kernel
589 command line parameters affecting CPU idle time management.
592 CPU idle time management entirely.  It does not prevent the idle loop from
593 running on idle CPUs, but it prevents the CPU idle time governors and drivers
594 from being invoked.  If it is added to the kernel command line, the idle loop
595 will ask the hardware to enter idle states on idle CPUs via the CPU architecture
598 processors implementing the architecture (i.e. CPU instruction set) in question,
599 however, so it is rather crude and not very energy-efficient.  For this reason,
609 The other kernel command line parameters controlling CPU idle time management
614 options related to CPU idle time management: ``idle=poll``, ``idle=halt``,
615 and ``idle=nomwait``.  The first two of them disable the ``acpi_idle`` and
617 ``CPUIdle`` subsystem to be disabled and makes the idle loop invoke the
618 architecture support code to deal with idle CPUs.  How it does that depends on
620 ``idle=halt`` case, the architecture support code will use the ``HLT``
622 and causes the hardware to attempt to enter the shallowest available idle state)
623 for this purpose, and if ``idle=poll`` is used, idle CPUs will execute a
625 that using ``idle=poll`` is somewhat drastic in many cases, as preventing idle
628 P-states (see |cpufreq|) that require any number of CPUs in a package to be
629 idle, so it very well may hurt single-thread computations performance as well as
630 energy-efficiency.  Thus using it for performance reasons may not be a good idea
633 The ``idle=nomwait`` option prevents the use of ``MWAIT`` instruction of
634 the CPU to enter idle states. When this option is used, the ``acpi_idle``
641 In addition to the architecture-level kernel command line options affecting CPU
642 idle time management, there are parameters affecting individual ``CPUIdle``
645 where ``<n>`` is an idle state index also used in the name of the given
647 `Representation of Idle States <idle-states-representation_>`_), causes the
649 idle states deeper than idle state ``<n>``.  In that case, they will never ask
650 for any of those idle states or expose them to the governor.  [The behavior of