diff options
| author | Mel Gorman <mgorman@techsingularity.net> | 2016-05-20 02:14:21 +0200 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2016-05-20 04:12:14 +0200 |
| commit | da838d4fcba675cbf864f225d76f970e91220ee6 (patch) | |
| tree | 165448850b88e5ce309ea92cc955c91836c868a4 /mm/page_alloc.c | |
| parent | mm, page_alloc: un-inline the bad part of free_pages_check (diff) | |
| download | linux-da838d4fcba675cbf864f225d76f970e91220ee6.tar.xz linux-da838d4fcba675cbf864f225d76f970e91220ee6.zip | |
mm, page_alloc: pull out side effects from free_pages_check
Check without side-effects should be easier to maintain. It also
removes the duplicated cpupid and flags reset done in !DEBUG_VM variant
of both free_pcp_prepare() and then bulkfree_pcp_prepare(). Finally, it
enables the next patch.
It shouldn't result in new branches, thanks to inlining of the check.
!DEBUG_VM bloat-o-meter:
add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-27 (-27)
function old new delta
__free_pages_ok 748 739 -9
free_pcppages_bulk 1403 1385 -18
DEBUG_VM:
add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-28 (-28)
function old new delta
free_pages_prepare 806 778 -28
This is also slightly faster because cpupid information is not set on
tail pages so we can avoid resets there.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'mm/page_alloc.c')
| -rw-r--r-- | mm/page_alloc.c | 13 |
1 files changed, 8 insertions, 5 deletions
diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d51543de1813..fea50b0cb405 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -833,11 +833,8 @@ static void free_pages_check_bad(struct page *page) static inline int free_pages_check(struct page *page) { - if (likely(page_expected_state(page, PAGE_FLAGS_CHECK_AT_FREE))) { - page_cpupid_reset_last(page); - page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; + if (likely(page_expected_state(page, PAGE_FLAGS_CHECK_AT_FREE))) return 0; - } /* Something has gone sideways, find it */ free_pages_check_bad(page); @@ -1078,7 +1075,11 @@ static bool free_pages_prepare(struct page *page, unsigned int order) for (i = 1; i < (1 << order); i++) { if (compound) bad += free_tail_pages_check(page, page + i); - bad += free_pages_check(page + i); + if (unlikely(free_pages_check(page + i))) { + bad++; + continue; + } + (page + i)->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; } } if (PageAnonHead(page)) @@ -1087,6 +1088,8 @@ static bool free_pages_prepare(struct page *page, unsigned int order) if (bad) return false; + page_cpupid_reset_last(page); + page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; reset_page_owner(page, order); if (!PageHighMem(page)) { |