1CONFIG_RCU_TRACE debugfs Files and Formats 2 3 4The rcutree and rcutiny implementations of RCU provide debugfs trace 5output that summarizes counters and state. This information is useful for 6debugging RCU itself, and can sometimes also help to debug abuses of RCU. 7The following sections describe the debugfs files and formats, first 8for rcutree and next for rcutiny. 9 10 11CONFIG_TREE_RCU and CONFIG_TREE_PREEMPT_RCU debugfs Files and Formats 12 13These implementations of RCU provides several debugfs files under the 14top-level directory "rcu": 15 16rcu/rcudata: 17 Displays fields in struct rcu_data. 18rcu/rcudata.csv: 19 Comma-separated values spreadsheet version of rcudata. 20rcu/rcugp: 21 Displays grace-period counters. 22rcu/rcuhier: 23 Displays the struct rcu_node hierarchy. 24rcu/rcu_pending: 25 Displays counts of the reasons rcu_pending() decided that RCU had 26 work to do. 27rcu/rcutorture: 28 Displays rcutorture test progress. 29rcu/rcuboost: 30 Displays RCU boosting statistics. Only present if 31 CONFIG_RCU_BOOST=y. 32 33The output of "cat rcu/rcudata" looks as follows: 34 35rcu_sched: 36 0 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=545/1/0 df=50 of=0 ri=0 ql=163 qs=NRW. kt=0/W/0 ktl=ebc3 b=10 ci=153737 co=0 ca=0 37 1 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=967/1/0 df=58 of=0 ri=0 ql=634 qs=NRW. kt=0/W/1 ktl=58c b=10 ci=191037 co=0 ca=0 38 2 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1081/1/0 df=175 of=0 ri=0 ql=74 qs=N.W. kt=0/W/2 ktl=da94 b=10 ci=75991 co=0 ca=0 39 3 c=20942 g=20943 pq=1 pgp=20942 qp=1 dt=1846/0/0 df=404 of=0 ri=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=72261 co=0 ca=0 40 4 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=369/1/0 df=83 of=0 ri=0 ql=48 qs=N.W. kt=0/W/4 ktl=e0e7 b=10 ci=128365 co=0 ca=0 41 5 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=381/1/0 df=64 of=0 ri=0 ql=169 qs=NRW. kt=0/W/5 ktl=fb2f b=10 ci=164360 co=0 ca=0 42 6 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1037/1/0 df=183 of=0 ri=0 ql=62 qs=N.W. kt=0/W/6 ktl=d2ad b=10 ci=65663 co=0 ca=0 43 7 c=20897 g=20897 pq=1 pgp=20896 qp=0 dt=1572/0/0 df=382 of=0 ri=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=75006 co=0 ca=0 44rcu_bh: 45 0 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=545/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/0 ktl=ebc3 b=10 ci=0 co=0 ca=0 46 1 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=967/1/0 df=3 of=0 ri=1 ql=0 qs=.... kt=0/W/1 ktl=58c b=10 ci=151 co=0 ca=0 47 2 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1081/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/2 ktl=da94 b=10 ci=0 co=0 ca=0 48 3 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1846/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=0 co=0 ca=0 49 4 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=369/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/4 ktl=e0e7 b=10 ci=0 co=0 ca=0 50 5 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=381/1/0 df=4 of=0 ri=1 ql=0 qs=.... kt=0/W/5 ktl=fb2f b=10 ci=0 co=0 ca=0 51 6 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1037/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/6 ktl=d2ad b=10 ci=0 co=0 ca=0 52 7 c=1474 g=1474 pq=1 pgp=1473 qp=0 dt=1572/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=0 co=0 ca=0 53 54The first section lists the rcu_data structures for rcu_sched, the second 55for rcu_bh. Note that CONFIG_TREE_PREEMPT_RCU kernels will have an 56additional section for rcu_preempt. Each section has one line per CPU, 57or eight for this 8-CPU system. The fields are as follows: 58 59o The number at the beginning of each line is the CPU number. 60 CPUs numbers followed by an exclamation mark are offline, 61 but have been online at least once since boot. There will be 62 no output for CPUs that have never been online, which can be 63 a good thing in the surprisingly common case where NR_CPUS is 64 substantially larger than the number of actual CPUs. 65 66o "c" is the count of grace periods that this CPU believes have 67 completed. Offlined CPUs and CPUs in dynticks idle mode may 68 lag quite a ways behind, for example, CPU 6 under "rcu_sched" 69 above, which has been offline through not quite 40,000 RCU grace 70 periods. It is not unusual to see CPUs lagging by thousands of 71 grace periods. 72 73o "g" is the count of grace periods that this CPU believes have 74 started. Again, offlined CPUs and CPUs in dynticks idle mode 75 may lag behind. If the "c" and "g" values are equal, this CPU 76 has already reported a quiescent state for the last RCU grace 77 period that it is aware of, otherwise, the CPU believes that it 78 owes RCU a quiescent state. 79 80o "pq" indicates that this CPU has passed through a quiescent state 81 for the current grace period. It is possible for "pq" to be 82 "1" and "c" different than "g", which indicates that although 83 the CPU has passed through a quiescent state, either (1) this 84 CPU has not yet reported that fact, (2) some other CPU has not 85 yet reported for this grace period, or (3) both. 86 87o "pgp" indicates which grace period the last-observed quiescent 88 state for this CPU corresponds to. This is important for handling 89 the race between CPU 0 reporting an extended dynticks-idle 90 quiescent state for CPU 1 and CPU 1 suddenly waking up and 91 reporting its own quiescent state. If CPU 1 was the last CPU 92 for the current grace period, then the CPU that loses this race 93 will attempt to incorrectly mark CPU 1 as having checked in for 94 the next grace period! 95 96o "qp" indicates that RCU still expects a quiescent state from 97 this CPU. Offlined CPUs and CPUs in dyntick idle mode might 98 well have qp=1, which is OK: RCU is still ignoring them. 99 100o "dt" is the current value of the dyntick counter that is incremented 101 when entering or leaving dynticks idle state, either by the 102 scheduler or by irq. This number is even if the CPU is in 103 dyntick idle mode and odd otherwise. The number after the first 104 "/" is the interrupt nesting depth when in dyntick-idle state, 105 or one greater than the interrupt-nesting depth otherwise. 106 The number after the second "/" is the NMI nesting depth. 107 108o "df" is the number of times that some other CPU has forced a 109 quiescent state on behalf of this CPU due to this CPU being in 110 dynticks-idle state. 111 112o "of" is the number of times that some other CPU has forced a 113 quiescent state on behalf of this CPU due to this CPU being 114 offline. In a perfect world, this might never happen, but it 115 turns out that offlining and onlining a CPU can take several grace 116 periods, and so there is likely to be an extended period of time 117 when RCU believes that the CPU is online when it really is not. 118 Please note that erring in the other direction (RCU believing a 119 CPU is offline when it is really alive and kicking) is a fatal 120 error, so it makes sense to err conservatively. 121 122o "ri" is the number of times that RCU has seen fit to send a 123 reschedule IPI to this CPU in order to get it to report a 124 quiescent state. 125 126o "ql" is the number of RCU callbacks currently residing on 127 this CPU. This is the total number of callbacks, regardless 128 of what state they are in (new, waiting for grace period to 129 start, waiting for grace period to end, ready to invoke). 130 131o "qs" gives an indication of the state of the callback queue 132 with four characters: 133 134 "N" Indicates that there are callbacks queued that are not 135 ready to be handled by the next grace period, and thus 136 will be handled by the grace period following the next 137 one. 138 139 "R" Indicates that there are callbacks queued that are 140 ready to be handled by the next grace period. 141 142 "W" Indicates that there are callbacks queued that are 143 waiting on the current grace period. 144 145 "D" Indicates that there are callbacks queued that have 146 already been handled by a prior grace period, and are 147 thus waiting to be invoked. Note that callbacks in 148 the process of being invoked are not counted here. 149 Callbacks in the process of being invoked are those 150 that have been removed from the rcu_data structures 151 queues by rcu_do_batch(), but which have not yet been 152 invoked. 153 154 If there are no callbacks in a given one of the above states, 155 the corresponding character is replaced by ".". 156 157o "kt" is the per-CPU kernel-thread state. The digit preceding 158 the first slash is zero if there is no work pending and 1 159 otherwise. The character between the first pair of slashes is 160 as follows: 161 162 "S" The kernel thread is stopped, in other words, all 163 CPUs corresponding to this rcu_node structure are 164 offline. 165 166 "R" The kernel thread is running. 167 168 "W" The kernel thread is waiting because there is no work 169 for it to do. 170 171 "O" The kernel thread is waiting because it has been 172 forced off of its designated CPU or because its 173 ->cpus_allowed mask permits it to run on other than 174 its designated CPU. 175 176 "Y" The kernel thread is yielding to avoid hogging CPU. 177 178 "?" Unknown value, indicates a bug. 179 180 The number after the final slash is the CPU that the kthread 181 is actually running on. 182 183 This field is displayed only for CONFIG_RCU_BOOST kernels. 184 185o "ktl" is the low-order 16 bits (in hexadecimal) of the count of 186 the number of times that this CPU's per-CPU kthread has gone 187 through its loop servicing invoke_rcu_cpu_kthread() requests. 188 189 This field is displayed only for CONFIG_RCU_BOOST kernels. 190 191o "b" is the batch limit for this CPU. If more than this number 192 of RCU callbacks is ready to invoke, then the remainder will 193 be deferred. 194 195o "ci" is the number of RCU callbacks that have been invoked for 196 this CPU. Note that ci+ql is the number of callbacks that have 197 been registered in absence of CPU-hotplug activity. 198 199o "co" is the number of RCU callbacks that have been orphaned due to 200 this CPU going offline. These orphaned callbacks have been moved 201 to an arbitrarily chosen online CPU. 202 203o "ca" is the number of RCU callbacks that have been adopted due to 204 other CPUs going offline. Note that ci+co-ca+ql is the number of 205 RCU callbacks registered on this CPU. 206 207There is also an rcu/rcudata.csv file with the same information in 208comma-separated-variable spreadsheet format. 209 210 211The output of "cat rcu/rcugp" looks as follows: 212 213rcu_sched: completed=33062 gpnum=33063 214rcu_bh: completed=464 gpnum=464 215 216Again, this output is for both "rcu_sched" and "rcu_bh". Note that 217kernels built with CONFIG_TREE_PREEMPT_RCU will have an additional 218"rcu_preempt" line. The fields are taken from the rcu_state structure, 219and are as follows: 220 221o "completed" is the number of grace periods that have completed. 222 It is comparable to the "c" field from rcu/rcudata in that a 223 CPU whose "c" field matches the value of "completed" is aware 224 that the corresponding RCU grace period has completed. 225 226o "gpnum" is the number of grace periods that have started. It is 227 comparable to the "g" field from rcu/rcudata in that a CPU 228 whose "g" field matches the value of "gpnum" is aware that the 229 corresponding RCU grace period has started. 230 231 If these two fields are equal (as they are for "rcu_bh" above), 232 then there is no grace period in progress, in other words, RCU 233 is idle. On the other hand, if the two fields differ (as they 234 do for "rcu_sched" above), then an RCU grace period is in progress. 235 236 237The output of "cat rcu/rcuhier" looks as follows, with very long lines: 238 239c=6902 g=6903 s=2 jfq=3 j=72c7 nfqs=13142/nfqsng=0(13142) fqlh=6 2401/1 ..>. 0:127 ^0 2413/3 ..>. 0:35 ^0 0/0 ..>. 36:71 ^1 0/0 ..>. 72:107 ^2 0/0 ..>. 108:127 ^3 2423/3f ..>. 0:5 ^0 2/3 ..>. 6:11 ^1 0/0 ..>. 12:17 ^2 0/0 ..>. 18:23 ^3 0/0 ..>. 24:29 ^4 0/0 ..>. 30:35 ^5 0/0 ..>. 36:41 ^0 0/0 ..>. 42:47 ^1 0/0 ..>. 48:53 ^2 0/0 ..>. 54:59 ^3 0/0 ..>. 60:65 ^4 0/0 ..>. 66:71 ^5 0/0 ..>. 72:77 ^0 0/0 ..>. 78:83 ^1 0/0 ..>. 84:89 ^2 0/0 ..>. 90:95 ^3 0/0 ..>. 96:101 ^4 0/0 ..>. 102:107 ^5 0/0 ..>. 108:113 ^0 0/0 ..>. 114:119 ^1 0/0 ..>. 120:125 ^2 0/0 ..>. 126:127 ^3 243rcu_bh: 244c=-226 g=-226 s=1 jfq=-5701 j=72c7 nfqs=88/nfqsng=0(88) fqlh=0 2450/1 ..>. 0:127 ^0 2460/3 ..>. 0:35 ^0 0/0 ..>. 36:71 ^1 0/0 ..>. 72:107 ^2 0/0 ..>. 108:127 ^3 2470/3f ..>. 0:5 ^0 0/3 ..>. 6:11 ^1 0/0 ..>. 12:17 ^2 0/0 ..>. 18:23 ^3 0/0 ..>. 24:29 ^4 0/0 ..>. 30:35 ^5 0/0 ..>. 36:41 ^0 0/0 ..>. 42:47 ^1 0/0 ..>. 48:53 ^2 0/0 ..>. 54:59 ^3 0/0 ..>. 60:65 ^4 0/0 ..>. 66:71 ^5 0/0 ..>. 72:77 ^0 0/0 ..>. 78:83 ^1 0/0 ..>. 84:89 ^2 0/0 ..>. 90:95 ^3 0/0 ..>. 96:101 ^4 0/0 ..>. 102:107 ^5 0/0 ..>. 108:113 ^0 0/0 ..>. 114:119 ^1 0/0 ..>. 120:125 ^2 0/0 ..>. 126:127 ^3 248 249This is once again split into "rcu_sched" and "rcu_bh" portions, 250and CONFIG_TREE_PREEMPT_RCU kernels will again have an additional 251"rcu_preempt" section. The fields are as follows: 252 253o "c" is exactly the same as "completed" under rcu/rcugp. 254 255o "g" is exactly the same as "gpnum" under rcu/rcugp. 256 257o "s" is the "signaled" state that drives force_quiescent_state()'s 258 state machine. 259 260o "jfq" is the number of jiffies remaining for this grace period 261 before force_quiescent_state() is invoked to help push things 262 along. Note that CPUs in dyntick-idle mode throughout the grace 263 period will not report on their own, but rather must be check by 264 some other CPU via force_quiescent_state(). 265 266o "j" is the low-order four hex digits of the jiffies counter. 267 Yes, Paul did run into a number of problems that turned out to 268 be due to the jiffies counter no longer counting. Why do you ask? 269 270o "nfqs" is the number of calls to force_quiescent_state() since 271 boot. 272 273o "nfqsng" is the number of useless calls to force_quiescent_state(), 274 where there wasn't actually a grace period active. This can 275 happen due to races. The number in parentheses is the difference 276 between "nfqs" and "nfqsng", or the number of times that 277 force_quiescent_state() actually did some real work. 278 279o "fqlh" is the number of calls to force_quiescent_state() that 280 exited immediately (without even being counted in nfqs above) 281 due to contention on ->fqslock. 282 283o Each element of the form "1/1 0:127 ^0" represents one struct 284 rcu_node. Each line represents one level of the hierarchy, from 285 root to leaves. It is best to think of the rcu_data structures 286 as forming yet another level after the leaves. Note that there 287 might be either one, two, or three levels of rcu_node structures, 288 depending on the relationship between CONFIG_RCU_FANOUT and 289 CONFIG_NR_CPUS. 290 291 o The numbers separated by the "/" are the qsmask followed 292 by the qsmaskinit. The qsmask will have one bit 293 set for each entity in the next lower level that 294 has not yet checked in for the current grace period. 295 The qsmaskinit will have one bit for each entity that is 296 currently expected to check in during each grace period. 297 The value of qsmaskinit is assigned to that of qsmask 298 at the beginning of each grace period. 299 300 For example, for "rcu_sched", the qsmask of the first 301 entry of the lowest level is 0x14, meaning that we 302 are still waiting for CPUs 2 and 4 to check in for the 303 current grace period. 304 305 o The characters separated by the ">" indicate the state 306 of the blocked-tasks lists. A "G" preceding the ">" 307 indicates that at least one task blocked in an RCU 308 read-side critical section blocks the current grace 309 period, while a "E" preceding the ">" indicates that 310 at least one task blocked in an RCU read-side critical 311 section blocks the current expedited grace period. 312 A "T" character following the ">" indicates that at 313 least one task is blocked within an RCU read-side 314 critical section, regardless of whether any current 315 grace period (expedited or normal) is inconvenienced. 316 A "." character appears if the corresponding condition 317 does not hold, so that "..>." indicates that no tasks 318 are blocked. In contrast, "GE>T" indicates maximal 319 inconvenience from blocked tasks. 320 321 o The numbers separated by the ":" are the range of CPUs 322 served by this struct rcu_node. This can be helpful 323 in working out how the hierarchy is wired together. 324 325 For example, the first entry at the lowest level shows 326 "0:5", indicating that it covers CPUs 0 through 5. 327 328 o The number after the "^" indicates the bit in the 329 next higher level rcu_node structure that this 330 rcu_node structure corresponds to. 331 332 For example, the first entry at the lowest level shows 333 "^0", indicating that it corresponds to bit zero in 334 the first entry at the middle level. 335 336 337The output of "cat rcu/rcu_pending" looks as follows: 338 339rcu_sched: 340 0 np=255892 qsp=53936 rpq=85 cbr=0 cng=14417 gpc=10033 gps=24320 nf=6445 nn=146741 341 1 np=261224 qsp=54638 rpq=33 cbr=0 cng=25723 gpc=16310 gps=2849 nf=5912 nn=155792 342 2 np=237496 qsp=49664 rpq=23 cbr=0 cng=2762 gpc=45478 gps=1762 nf=1201 nn=136629 343 3 np=236249 qsp=48766 rpq=98 cbr=0 cng=286 gpc=48049 gps=1218 nf=207 nn=137723 344 4 np=221310 qsp=46850 rpq=7 cbr=0 cng=26 gpc=43161 gps=4634 nf=3529 nn=123110 345 5 np=237332 qsp=48449 rpq=9 cbr=0 cng=54 gpc=47920 gps=3252 nf=201 nn=137456 346 6 np=219995 qsp=46718 rpq=12 cbr=0 cng=50 gpc=42098 gps=6093 nf=4202 nn=120834 347 7 np=249893 qsp=49390 rpq=42 cbr=0 cng=72 gpc=38400 gps=17102 nf=41 nn=144888 348rcu_bh: 349 0 np=146741 qsp=1419 rpq=6 cbr=0 cng=6 gpc=0 gps=0 nf=2 nn=145314 350 1 np=155792 qsp=12597 rpq=3 cbr=0 cng=0 gpc=4 gps=8 nf=3 nn=143180 351 2 np=136629 qsp=18680 rpq=1 cbr=0 cng=0 gpc=7 gps=6 nf=0 nn=117936 352 3 np=137723 qsp=2843 rpq=0 cbr=0 cng=0 gpc=10 gps=7 nf=0 nn=134863 353 4 np=123110 qsp=12433 rpq=0 cbr=0 cng=0 gpc=4 gps=2 nf=0 nn=110671 354 5 np=137456 qsp=4210 rpq=1 cbr=0 cng=0 gpc=6 gps=5 nf=0 nn=133235 355 6 np=120834 qsp=9902 rpq=2 cbr=0 cng=0 gpc=6 gps=3 nf=2 nn=110921 356 7 np=144888 qsp=26336 rpq=0 cbr=0 cng=0 gpc=8 gps=2 nf=0 nn=118542 357 358As always, this is once again split into "rcu_sched" and "rcu_bh" 359portions, with CONFIG_TREE_PREEMPT_RCU kernels having an additional 360"rcu_preempt" section. The fields are as follows: 361 362o "np" is the number of times that __rcu_pending() has been invoked 363 for the corresponding flavor of RCU. 364 365o "qsp" is the number of times that the RCU was waiting for a 366 quiescent state from this CPU. 367 368o "rpq" is the number of times that the CPU had passed through 369 a quiescent state, but not yet reported it to RCU. 370 371o "cbr" is the number of times that this CPU had RCU callbacks 372 that had passed through a grace period, and were thus ready 373 to be invoked. 374 375o "cng" is the number of times that this CPU needed another 376 grace period while RCU was idle. 377 378o "gpc" is the number of times that an old grace period had 379 completed, but this CPU was not yet aware of it. 380 381o "gps" is the number of times that a new grace period had started, 382 but this CPU was not yet aware of it. 383 384o "nf" is the number of times that this CPU suspected that the 385 current grace period had run for too long, and thus needed to 386 be forced. 387 388 Please note that "forcing" consists of sending resched IPIs 389 to holdout CPUs. If that CPU really still is in an old RCU 390 read-side critical section, then we really do have to wait for it. 391 The assumption behing "forcing" is that the CPU is not still in 392 an old RCU read-side critical section, but has not yet responded 393 for some other reason. 394 395o "nn" is the number of times that this CPU needed nothing. Alert 396 readers will note that the rcu "nn" number for a given CPU very 397 closely matches the rcu_bh "np" number for that same CPU. This 398 is due to short-circuit evaluation in rcu_pending(). 399 400 401The output of "cat rcu/rcutorture" looks as follows: 402 403rcutorture test sequence: 0 (test in progress) 404rcutorture update version number: 615 405 406The first line shows the number of rcutorture tests that have completed 407since boot. If a test is currently running, the "(test in progress)" 408string will appear as shown above. The second line shows the number of 409update cycles that the current test has started, or zero if there is 410no test in progress. 411 412 413The output of "cat rcu/rcuboost" looks as follows: 414 4150:5 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=2f95 bt=300f 416 balk: nt=0 egt=989 bt=0 nb=0 ny=0 nos=16 4176:7 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=2f95 bt=300f 418 balk: nt=0 egt=225 bt=0 nb=0 ny=0 nos=6 419 420This information is output only for rcu_preempt. Each two-line entry 421corresponds to a leaf rcu_node strcuture. The fields are as follows: 422 423o "n:m" is the CPU-number range for the corresponding two-line 424 entry. In the sample output above, the first entry covers 425 CPUs zero through five and the second entry covers CPUs 6 426 and 7. 427 428o "tasks=TNEB" gives the state of the various segments of the 429 rnp->blocked_tasks list: 430 431 "T" This indicates that there are some tasks that blocked 432 while running on one of the corresponding CPUs while 433 in an RCU read-side critical section. 434 435 "N" This indicates that some of the blocked tasks are preventing 436 the current normal (non-expedited) grace period from 437 completing. 438 439 "E" This indicates that some of the blocked tasks are preventing 440 the current expedited grace period from completing. 441 442 "B" This indicates that some of the blocked tasks are in 443 need of RCU priority boosting. 444 445 Each character is replaced with "." if the corresponding 446 condition does not hold. 447 448o "kt" is the state of the RCU priority-boosting kernel 449 thread associated with the corresponding rcu_node structure. 450 The state can be one of the following: 451 452 "S" The kernel thread is stopped, in other words, all 453 CPUs corresponding to this rcu_node structure are 454 offline. 455 456 "R" The kernel thread is running. 457 458 "W" The kernel thread is waiting because there is no work 459 for it to do. 460 461 "Y" The kernel thread is yielding to avoid hogging CPU. 462 463 "?" Unknown value, indicates a bug. 464 465o "ntb" is the number of tasks boosted. 466 467o "neb" is the number of tasks boosted in order to complete an 468 expedited grace period. 469 470o "nnb" is the number of tasks boosted in order to complete a 471 normal (non-expedited) grace period. When boosting a task 472 that was blocking both an expedited and a normal grace period, 473 it is counted against the expedited total above. 474 475o "j" is the low-order 16 bits of the jiffies counter in 476 hexadecimal. 477 478o "bt" is the low-order 16 bits of the value that the jiffies 479 counter will have when we next start boosting, assuming that 480 the current grace period does not end beforehand. This is 481 also in hexadecimal. 482 483o "balk: nt" counts the number of times we didn't boost (in 484 other words, we balked) even though it was time to boost because 485 there were no blocked tasks to boost. This situation occurs 486 when there is one blocked task on one rcu_node structure and 487 none on some other rcu_node structure. 488 489o "egt" counts the number of times we balked because although 490 there were blocked tasks, none of them were blocking the 491 current grace period, whether expedited or otherwise. 492 493o "bt" counts the number of times we balked because boosting 494 had already been initiated for the current grace period. 495 496o "nb" counts the number of times we balked because there 497 was at least one task blocking the current non-expedited grace 498 period that never had blocked. If it is already running, it 499 just won't help to boost its priority! 500 501o "ny" counts the number of times we balked because it was 502 not yet time to start boosting. 503 504o "nos" counts the number of times we balked for other 505 reasons, e.g., the grace period ended first. 506 507 508CONFIG_TINY_RCU and CONFIG_TINY_PREEMPT_RCU debugfs Files and Formats 509 510These implementations of RCU provides a single debugfs file under the 511top-level directory RCU, namely rcu/rcudata, which displays fields in 512rcu_bh_ctrlblk, rcu_sched_ctrlblk and, for CONFIG_TINY_PREEMPT_RCU, 513rcu_preempt_ctrlblk. 514 515The output of "cat rcu/rcudata" is as follows: 516 517rcu_preempt: qlen=24 gp=1097669 g197/p197/c197 tasks=... 518 ttb=. btg=no ntb=184 neb=0 nnb=183 j=01f7 bt=0274 519 normal balk: nt=1097669 gt=0 bt=371 b=0 ny=25073378 nos=0 520 exp balk: bt=0 nos=0 521rcu_sched: qlen: 0 522rcu_bh: qlen: 0 523 524This is split into rcu_preempt, rcu_sched, and rcu_bh sections, with the 525rcu_preempt section appearing only in CONFIG_TINY_PREEMPT_RCU builds. 526The last three lines of the rcu_preempt section appear only in 527CONFIG_RCU_BOOST kernel builds. The fields are as follows: 528 529o "qlen" is the number of RCU callbacks currently waiting either 530 for an RCU grace period or waiting to be invoked. This is the 531 only field present for rcu_sched and rcu_bh, due to the 532 short-circuiting of grace period in those two cases. 533 534o "gp" is the number of grace periods that have completed. 535 536o "g197/p197/c197" displays the grace-period state, with the 537 "g" number being the number of grace periods that have started 538 (mod 256), the "p" number being the number of grace periods 539 that the CPU has responded to (also mod 256), and the "c" 540 number being the number of grace periods that have completed 541 (once again mode 256). 542 543 Why have both "gp" and "g"? Because the data flowing into 544 "gp" is only present in a CONFIG_RCU_TRACE kernel. 545 546o "tasks" is a set of bits. The first bit is "T" if there are 547 currently tasks that have recently blocked within an RCU 548 read-side critical section, the second bit is "N" if any of the 549 aforementioned tasks are blocking the current RCU grace period, 550 and the third bit is "E" if any of the aforementioned tasks are 551 blocking the current expedited grace period. Each bit is "." 552 if the corresponding condition does not hold. 553 554o "ttb" is a single bit. It is "B" if any of the blocked tasks 555 need to be priority boosted and "." otherwise. 556 557o "btg" indicates whether boosting has been carried out during 558 the current grace period, with "exp" indicating that boosting 559 is in progress for an expedited grace period, "no" indicating 560 that boosting has not yet started for a normal grace period, 561 "begun" indicating that boosting has bebug for a normal grace 562 period, and "done" indicating that boosting has completed for 563 a normal grace period. 564 565o "ntb" is the total number of tasks subjected to RCU priority boosting 566 periods since boot. 567 568o "neb" is the number of expedited grace periods that have had 569 to resort to RCU priority boosting since boot. 570 571o "nnb" is the number of normal grace periods that have had 572 to resort to RCU priority boosting since boot. 573 574o "j" is the low-order 16 bits of the jiffies counter in hexadecimal. 575 576o "bt" is the low-order 16 bits of the value that the jiffies counter 577 will have at the next time that boosting is scheduled to begin. 578 579o In the line beginning with "normal balk", the fields are as follows: 580 581 o "nt" is the number of times that the system balked from 582 boosting because there were no blocked tasks to boost. 583 Note that the system will balk from boosting even if the 584 grace period is overdue when the currently running task 585 is looping within an RCU read-side critical section. 586 There is no point in boosting in this case, because 587 boosting a running task won't make it run any faster. 588 589 o "gt" is the number of times that the system balked 590 from boosting because, although there were blocked tasks, 591 none of them were preventing the current grace period 592 from completing. 593 594 o "bt" is the number of times that the system balked 595 from boosting because boosting was already in progress. 596 597 o "b" is the number of times that the system balked from 598 boosting because boosting had already completed for 599 the grace period in question. 600 601 o "ny" is the number of times that the system balked from 602 boosting because it was not yet time to start boosting 603 the grace period in question. 604 605 o "nos" is the number of times that the system balked from 606 boosting for inexplicable ("not otherwise specified") 607 reasons. This can actually happen due to races involving 608 increments of the jiffies counter. 609 610o In the line beginning with "exp balk", the fields are as follows: 611 612 o "bt" is the number of times that the system balked from 613 boosting because there were no blocked tasks to boost. 614 615 o "nos" is the number of times that the system balked from 616 boosting for inexplicable ("not otherwise specified") 617 reasons. 618