diff options
| author | Michel Lespinasse <walken@google.com> | 2013-05-07 15:46:00 +0200 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2013-05-07 16:20:17 +0200 |
| commit | 25c39325968bbcebe6cd2a1991228c9dfb48d655 (patch) | |
| tree | 627e32434338adcf4f7583219e9b4014b738dcc3 /lib | |
| parent | rwsem: implement support for write lock stealing on the fastpath (diff) | |
| download | linux-25c39325968bbcebe6cd2a1991228c9dfb48d655.tar.xz linux-25c39325968bbcebe6cd2a1991228c9dfb48d655.zip | |
rwsem: do not block readers at head of queue if other readers are active
This change fixes a race condition where a reader might determine it
needs to block, but by the time it acquires the wait_lock the rwsem has
active readers and no queued waiters.
In this situation the reader can run in parallel with the existing
active readers; it does not need to block until the active readers
complete.
Thanks to Peter Hurley for noticing this possible race.
Signed-off-by: Michel Lespinasse <walken@google.com>
Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'lib')
| -rw-r--r-- | lib/rwsem.c | 10 |
1 files changed, 8 insertions, 2 deletions
diff --git a/lib/rwsem.c b/lib/rwsem.c index bbe48c04f363..61f91ca75e40 100644 --- a/lib/rwsem.c +++ b/lib/rwsem.c @@ -163,8 +163,14 @@ struct rw_semaphore __sched *rwsem_down_read_failed(struct rw_semaphore *sem) /* we're now waiting on the lock, but no longer actively locking */ count = rwsem_atomic_update(adjustment, sem); - /* If there are no active locks, wake the front queued process(es). */ - if (!(count & RWSEM_ACTIVE_MASK)) + /* If there are no active locks, wake the front queued process(es). + * + * If there are no writers and we are first in the queue, + * wake our own waiter to join the existing active readers ! + */ + if (count == RWSEM_WAITING_BIAS || + (count > RWSEM_WAITING_BIAS && + adjustment != -RWSEM_ACTIVE_READ_BIAS)) sem = __rwsem_do_wake(sem, RWSEM_WAKE_ANY); raw_spin_unlock_irq(&sem->wait_lock); |