diff options
| author | Davidlohr Bueso <dave@stgolabs.net> | 2017-02-23 00:44:55 +0100 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2017-02-23 01:41:29 +0100 |
| commit | 46acef048a6568ba490f636db8682a1461ed223c (patch) | |
| tree | 7efc7b456db2e0b3df833f895d409bf20efa8646 /mm/compaction.c | |
| parent | mm: page_alloc: skip over regions of invalid pfns where possible (diff) | |
| download | linux-46acef048a6568ba490f636db8682a1461ed223c.tar.xz linux-46acef048a6568ba490f636db8682a1461ed223c.zip | |
mm,compaction: serialize waitqueue_active() checks
Without a memory barrier, the following race can occur with a high-order
allocation:
wakeup_kcompactd(order == 1) kcompactd()
[L] waitqueue_active(kcompactd_wait)
[S] prepare_to_wait_event(kcompactd_wait)
[L] (kcompactd_max_order == 0)
[S] kcompactd_max_order = order; schedule()
Where the waitqueue_active() check is speculatively re-ordered to before
setting the actual condition (max_order), not seeing the threads that's
going to block; making us miss a wakeup. There are a couple of options
to fix this, including calling wq_has_sleepers() which adds a full
barrier, or unconditionally doing the wake_up_interruptible() and
serialize on the q->lock. However, to make use of the control
dependency, we just need to add L->L guarantees.
While this bug is theoretical, there have been other offenders of the
lockless waitqueue_active() in the past -- this is also documented in
the call itself.
Link: http://lkml.kernel.org/r/1483975528-24342-1-git-send-email-dave@stgolabs.net
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'mm/compaction.c')
| -rw-r--r-- | mm/compaction.c | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/mm/compaction.c b/mm/compaction.c index c6178bbd3e04..0aa2757399ee 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1966,6 +1966,13 @@ void wakeup_kcompactd(pg_data_t *pgdat, int order, int classzone_idx) if (pgdat->kcompactd_max_order < order) pgdat->kcompactd_max_order = order; + /* + * Pairs with implicit barrier in wait_event_freezable() + * such that wakeups are not missed in the lockless + * waitqueue_active() call. + */ + smp_acquire__after_ctrl_dep(); + if (pgdat->kcompactd_classzone_idx > classzone_idx) pgdat->kcompactd_classzone_idx = classzone_idx; |