Documentation/RCU/stallwarn.rst

7 This document first discusses what sorts of issues RCU's CPU stall
10 this document explains the stall detector's "splat" format.
16 So your kernel printed an RCU CPU stall warning.  The next question is
17 "What caused it?"  The following problems can result in RCU CPU stall
37 	RCU CPU stall warning messages.  Especially if you have added
54 	memory, you might see stall-warning messages.
62 	CONFIG_PREEMPT_RCU case, you might see stall-warning
77 	RCU CPU stall warnings.
79 -	Testing a workload on a fast system, tuning the stall-warning
80 	timeout down to just barely avoid RCU CPU stall warnings, and then
81 	running the same workload with the same stall-warning timeout on a
88 	result in RCU CPU stall warnings for CONFIG_NO_HZ_COMMON=n kernels.
100 	believes that the RCU CPU stall-warning timeout has been exceeded
122 	of RCU CPU stall warnings, eventually leading the realization
126 CPU stall warning.  Note that SRCU does *not* have CPU stall warnings.
128 in progress.  No grace period, no CPU stall warnings.
130 To diagnose the cause of the stall, inspect the stack traces.
132 If you have a series of stall warnings from a single extended stall,
133 comparing the stack traces can often help determine where the stall
136 If you can reliably trigger the stall, ftrace can be quite helpful.
147 CPU stall detector, which detects conditions that unduly delay RCU grace
148 periods.  This module parameter enables CPU stall detection by default,
150 The stall detector's idea of what constitutes "unduly delayed" is
158 	issues an RCU CPU stall warning.  This time period is normally
164 	So if you are 10 seconds into a 40-second stall, setting this
166 	*next* stall, or the following warning for the current stall
167 	(assuming the stall lasts long enough).  It will not affect the
168 	timing of the next warning for the current stall.
179 	grace period until it issues an RCU CPU stall warning. This time
187 	are in a current stall cycle, setting it to a new value will change
188 	the timeout for the -next- stall.
199 	giving an RCU CPU stall warning message.  (This is a cpp
205 	The CPU stall detector tries to make the offending CPU print its
207 	However, if the offending CPU does not detect its own stall in
217 	RCU-tasks-trace stall warning intervals.  A value of zero or less
218 	suppresses RCU-tasks stall warnings.  A positive value sets the
219 	stall-warning interval in seconds.  An RCU-tasks stall warning
227 	An RCU-tasks-trace stall warning starts (and continues) similarly:
244 causing stalls, and that the stall was affecting RCU-sched.  This message
248 possible for an rcu_state stall to be caused by both CPUs *and* tasks,
251 in a self-detected stall.
275 across repeated stall-warning messages, it is possible that RCU's softirq
285 The "detected by" line indicates which CPU detected the stall (in this
291 If the grace period ends just as the stall warning starts printing,
292 there will be a spurious stall-warning message, which will include
298 possible for a zero-jiffy stall to be flagged in this case, depending
299 on how the stall warning and the grace-period initialization happen to
305 grace period has nevertheless failed to end, the stall-warning splat
318 the stall warning, as was the case in the "All QSes seen" line above,
325 in RCU CPU stall warnings even when all CPUs and tasks have passed
349 change on successive RCU CPU stall warnings, there is further reason to
353 involved in the stall.  These stack traces can help you locate the cause
354 of the stall, keeping in mind that the CPU detecting the stall will have
355 an interrupt frame that is mainly devoted to detecting the stall.
361 If a stall lasts long enough, multiple stall-warning messages will
365 of the stall and the first message.  It can be helpful to compare the
372 If an expedited grace period detects a stall, it will place a message
398 It is entirely possible to see stall warnings from normal and from
406 is supplied with each RCU CPU stall warning::
429              snapshot time point    1st-stall                  2nd-stall
489    If this type of RCU CPU stall warning can be reproduced, you can