diff options
| author | Omar Sandoval <osandov@fb.com> | 2017-01-18 20:55:21 +0100 |
|---|---|---|
| committer | Jens Axboe <axboe@fb.com> | 2017-01-18 21:41:49 +0100 |
| commit | f66227de5924ed0fde1823f5cbc4d8b8f45faaa2 (patch) | |
| tree | 0de01e70f6d96b76b1b38e66a0b4bef2231b9634 /lib/sbitmap.c | |
| parent | blk-cgroup: ensure that we clear the stop bit on quiesced queues (diff) | |
| download | linux-f66227de5924ed0fde1823f5cbc4d8b8f45faaa2.tar.xz linux-f66227de5924ed0fde1823f5cbc4d8b8f45faaa2.zip | |
sbitmap: use smp_mb__after_atomic() in sbq_wake_up()
We always do an atomic clear_bit() right before we call sbq_wake_up(),
so we can use smp_mb__after_atomic(). While we're here, comment the
memory barriers in here a little more.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Diffstat (limited to 'lib/sbitmap.c')
| -rw-r--r-- | lib/sbitmap.c | 13 |
1 files changed, 10 insertions, 3 deletions
diff --git a/lib/sbitmap.c b/lib/sbitmap.c index 2cecf05c82fd..df4e472df8a3 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -299,8 +299,14 @@ static void sbq_wake_up(struct sbitmap_queue *sbq) struct sbq_wait_state *ws; int wait_cnt; - /* Ensure that the wait list checks occur after clear_bit(). */ - smp_mb(); + /* + * Pairs with the memory barrier in set_current_state() to ensure the + * proper ordering of clear_bit()/waitqueue_active() in the waker and + * test_and_set_bit()/prepare_to_wait()/finish_wait() in the waiter. See + * the comment on waitqueue_active(). This is __after_atomic because we + * just did clear_bit() in the caller. + */ + smp_mb__after_atomic(); ws = sbq_wake_ptr(sbq); if (!ws) @@ -331,7 +337,8 @@ void sbitmap_queue_wake_all(struct sbitmap_queue *sbq) int i, wake_index; /* - * Make sure all changes prior to this are visible from other CPUs. + * Pairs with the memory barrier in set_current_state() like in + * sbq_wake_up(). */ smp_mb(); wake_index = atomic_read(&sbq->wake_index); |