diff options
author | Paul E. McKenney <paulmck@linux.vnet.ibm.com> | 2009-11-02 22:52:28 +0100 |
---|---|---|
committer | Ingo Molnar <mingo@elte.hu> | 2009-11-10 04:11:54 +0100 |
commit | d09b62dfa336447c52a5ec9bb88adbc479b0f3b8 (patch) | |
tree | 70a002fed2e0471def01ea3c166137449cb1527d /kernel/rcutree.h | |
parent | rcu: Prepare for synchronization fixes: clean up for non-NO_HZ handling of ->... (diff) | |
download | linux-d09b62dfa336447c52a5ec9bb88adbc479b0f3b8.tar.xz linux-d09b62dfa336447c52a5ec9bb88adbc479b0f3b8.zip |
rcu: Fix synchronization for rcu_process_gp_end() uses of ->completed counter
Impose a clear locking design on the rcu_process_gp_end()
function's use of the ->completed counter. This is done by
creating a ->completed field in the rcu_node structure, which
can safely be accessed under the protection of that structure's
lock. Performance and scalability are maintained by using a
form of double-checked locking, so that rcu_process_gp_end()
only acquires the leaf rcu_node structure's ->lock if a grace
period has recently ended.
This fix reduces rcutorture failure rate by at least two orders
of magnitude under heavy stress with force_quiescent_state()
being invoked artificially often. Without this fix,
unsynchronized access to the ->completed field can cause
rcu_process_gp_end() to advance callbacks whose grace period has
not yet expired. (Bad idea!)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
Cc: <stable@kernel.org> # .32.x
LKML-Reference: <12571987494069-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Diffstat (limited to 'kernel/rcutree.h')
-rw-r--r-- | kernel/rcutree.h | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/kernel/rcutree.h b/kernel/rcutree.h index 8a4c1650ad8d..c1891c3cae63 100644 --- a/kernel/rcutree.h +++ b/kernel/rcutree.h @@ -84,6 +84,9 @@ struct rcu_node { long gpnum; /* Current grace period for this node. */ /* This will either be equal to or one */ /* behind the root rcu_node's gpnum. */ + long completed; /* Last grace period completed for this node. */ + /* This will either be equal to or one */ + /* behind the root rcu_node's gpnum. */ unsigned long qsmask; /* CPUs or groups that need to switch in */ /* order for current grace period to proceed.*/ /* In leaf rcu_node, each bit corresponds to */ |