diff options
| author | Paul E. McKenney <paulmck@linux.ibm.com> | 2019-04-02 00:12:47 +0200 |
|---|---|---|
| committer | Paul E. McKenney <paulmck@linux.ibm.com> | 2019-05-25 23:50:48 +0200 |
| commit | 385b599e8c04fa843c4d7f785478827cc512d720 (patch) | |
| tree | 31ec5afef56e23943efcbf6c652059fd5e5cf73f /kernel/rcu | |
| parent | rcu: Only do rcu_read_unlock_special() wakeups if expedited (diff) | |
| download | linux-385b599e8c04fa843c4d7f785478827cc512d720.tar.xz linux-385b599e8c04fa843c4d7f785478827cc512d720.zip | |
rcu: Allow rcu_read_unlock_special() to raise_softirq() if in_irq()
When running in an interrupt handler, raise_softirq() and
raise_softirq_irqoff() have extremely low overhead: They simply set a
bit in a per-CPU mask, which is checked upon exit from that interrupt
handler. Therefore, if rcu_read_unlock_special() is invoked within an
interrupt handler and RCU_SOFTIRQ is in use, this commit make use of
raise_softirq_irqoff() even if there is no expedited grace period in
flight and even if this is not a nohz_full CPU.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Diffstat (limited to 'kernel/rcu')
| -rw-r--r-- | kernel/rcu/tree_plugin.h | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index d15cdab6aeb4..e1005f5e8094 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -615,7 +615,7 @@ static void rcu_read_unlock_special(struct task_struct *t) (rdp->grpmask & rnp->expmask) || tick_nohz_full_cpu(rdp->cpu); // Need to defer quiescent state until everything is enabled. - if (exp && irqs_were_disabled && use_softirq && + if ((exp || in_irq()) && irqs_were_disabled && use_softirq && (in_irq() || !t->rcu_read_unlock_special.b.deferred_qs)) { // Using softirq, safe to awaken, and we get // no help from enabling irqs, unlike bh/preempt. |