summaryrefslogtreecommitdiffstats
path: root/kernel/rcu/update.c
diff options
context:
space:
mode:
authorPaul E. McKenney <paulmck@linux.vnet.ibm.com>2014-08-15 01:01:53 +0200
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>2014-09-08 01:27:34 +0200
commit1d082fd061884a587c490c4fc8a2056ce1e47624 (patch)
treea8f715a674a036b1b7500a16ea11381d40659e42 /kernel/rcu/update.c
parentrcu: Additional information on RCU-tasks stall-warning messages (diff)
downloadlinux-1d082fd061884a587c490c4fc8a2056ce1e47624.tar.xz
linux-1d082fd061884a587c490c4fc8a2056ce1e47624.zip
rcu: Remove local_irq_disable() in rcu_preempt_note_context_switch()
The rcu_preempt_note_context_switch() function is on a scheduling fast path, so it would be good to avoid disabling irqs. The reason that irqs are disabled is to synchronize process-level and irq-handler access to the task_struct ->rcu_read_unlock_special bitmask. This commit therefore makes ->rcu_read_unlock_special instead be a union of bools with a short allowing single-access checks in RCU's __rcu_read_unlock(). This results in the process-level and irq-handler accesses being simple loads and stores, so that irqs need no longer be disabled. This commit therefore removes the irq disabling from rcu_preempt_note_context_switch(). Reported-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Diffstat (limited to 'kernel/rcu/update.c')
-rw-r--r--kernel/rcu/update.c2
1 files changed, 1 insertions, 1 deletions
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 9487b4898e51..6fb911558562 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -93,7 +93,7 @@ void __rcu_read_unlock(void)
barrier(); /* critical section before exit code. */
t->rcu_read_lock_nesting = INT_MIN;
barrier(); /* assign before ->rcu_read_unlock_special load */
- if (unlikely(ACCESS_ONCE(t->rcu_read_unlock_special)))
+ if (unlikely(ACCESS_ONCE(t->rcu_read_unlock_special.s)))
rcu_read_unlock_special(t);
barrier(); /* ->rcu_read_unlock_special load before assign */
t->rcu_read_lock_nesting = 0;