summaryrefslogtreecommitdiffstats
path: root/kernel
diff options
context:
space:
mode:
authorNicholas Mc Guire <der.herr@hofr.at>2015-01-23 12:41:47 +0100
committerIngo Molnar <mingo@kernel.org>2015-02-04 07:57:37 +0100
commit7c34e3180a01c800a40bc8535654d5735802fc1b (patch)
tree7b2b9408d49b48b4aed1ae94619112da0569ee7e /kernel
parentsched/completion: Remove unnecessary ->wait.lock serialization when reading c... (diff)
downloadlinux-7c34e3180a01c800a40bc8535654d5735802fc1b.tar.xz
linux-7c34e3180a01c800a40bc8535654d5735802fc1b.zip
sched/completion: Add lock-free checking of the blocking case
The "thread would block" case can be checked without grabbing ->wait.lock. [ If the check does not return early then grab the lock and recheck. A memory barrier is not needed as complete() and complete_all() imply a barrier. The ACCESS_ONCE() is needed for calls in a loop that, if inlined, could optimize out the re-fetching of x->done. ] Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/1422013307-13200-1-git-send-email-der.herr@hofr.at Signed-off-by: Ingo Molnar <mingo@kernel.org>
Diffstat (limited to 'kernel')
-rw-r--r--kernel/sched/completion.c9
1 files changed, 9 insertions, 0 deletions
diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c
index 9d1fe32da232..7052d3fd4e7b 100644
--- a/kernel/sched/completion.c
+++ b/kernel/sched/completion.c
@@ -268,6 +268,15 @@ bool try_wait_for_completion(struct completion *x)
unsigned long flags;
int ret = 1;
+ /*
+ * Since x->done will need to be locked only
+ * in the non-blocking case, we check x->done
+ * first without taking the lock so we can
+ * return early in the blocking case.
+ */
+ if (!ACCESS_ONCE(x->done))
+ return 0;
+
spin_lock_irqsave(&x->wait.lock, flags);
if (!x->done)
ret = 0;