kernel/rcu/rcu_segcblist.c

97 /* Return number of callbacks in segmented callback list by summing seglen. */
146  * field to disagree with the actual number of callbacks on the structure.
155  * This can of course race with both queuing and invoking of callbacks.
157  * rcu_barrier() failing to IPI a CPU that actually had callbacks queued
168  * CASE 1: Suppose that CPU 0 has no callbacks queued, but invokes
226  * callbacks on the structure.  This increase is fully ordered with respect
253  * Disable the specified rcu_segcblist structure, so that callbacks can
275  * Does the specified rcu_segcblist structure contain callbacks that
285  * Does the specified rcu_segcblist structure contain callbacks that
337  * for rcu_barrier() to sometimes post callbacks needlessly, but
357  * will wait for all prior callbacks, NOT necessarily for a grace
381  * Extract only those callbacks ready to be invoked from the specified
404  * Extract only those callbacks still pending (not yet ready to be
407  * about any callbacks that might have been partway done waiting for
439  * Move callbacks from the specified rcu_cblist to the beginning of the
440  * done-callbacks segment of the specified rcu_segcblist.
448 		return; /* No callbacks to move. */  in rcu_segcblist_insert_done_cbs()
462  * Move callbacks from the specified rcu_cblist to the end of the
463  * new-callbacks segment of the specified rcu_segcblist.
477  * Advance the callbacks in the specified rcu_segcblist structure based
489 	 * Find all callbacks whose ->gp_seq numbers indicate that they  in rcu_segcblist_advance()
499 	/* If no callbacks moved, nothing more need be done. */  in rcu_segcblist_advance()
508 	 * Callbacks moved, so there might be an empty RCU_WAIT_TAIL  in rcu_segcblist_advance()
515 			break;  /* No more callbacks. */  in rcu_segcblist_advance()
523  * "Accelerate" callbacks based on more-accurate grace-period information.
525  * ends of grace periods, and that callbacks are posted locally.  This in
526  * turn means that the callbacks must be labelled conservatively early
529  * available, previously posted callbacks can be "accelerated", marking
533  * grace-period sequence number seq at which new callbacks would become
534  * ready to invoke.  Returns true if there are callbacks that won't be
546 	 * Find the segment preceding the oldest segment of callbacks  in rcu_segcblist_accelerate()
550 	 * callbacks in the RCU_NEXT_TAIL segment, and assigned "seq"  in rcu_segcblist_accelerate()
559 	 * If all the segments contain callbacks that correspond to  in rcu_segcblist_accelerate()
563 	 * the non-done segments contain callbacks that really are  in rcu_segcblist_accelerate()
567 	 * Also advance to the oldest segment of callbacks whose  in rcu_segcblist_accelerate()
572 	 * containing older callbacks) will be unaffected, and their  in rcu_segcblist_accelerate()
587 	 * Merge all later callbacks, including newly arrived callbacks,  in rcu_segcblist_accelerate()
590 	 * where there were no pending callbacks in the rcu_segcblist  in rcu_segcblist_accelerate()
603  * callbacks from the source get to start over.  It is best to