diff options
| author | Mel Gorman <mgorman@techsingularity.net> | 2022-03-22 22:43:33 +0100 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2022-03-22 23:57:06 +0100 |
| commit | 35b6d770e6334aa470080570f0f81c8b74a07afd (patch) | |
| tree | bfd767977bc24462df79ffa4acb8abfd0846d4f2 /mm/page_alloc.c | |
| parent | mm/page_alloc: fetch the correct pcp buddy during bulk free (diff) | |
| download | linux-35b6d770e6334aa470080570f0f81c8b74a07afd.tar.xz linux-35b6d770e6334aa470080570f0f81c8b74a07afd.zip | |
mm/page_alloc: track range of active PCP lists during bulk free
free_pcppages_bulk() frees pages in a round-robin fashion. Originally,
this was dealing only with migratetypes but storing high-order pages
means that there can be many more empty lists that are uselessly
checked. Track the minimum and maximum active pindex to reduce the
search space.
Link: https://lkml.kernel.org/r/20220217002227.5739-3-mgorman@techsingularity.net
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Tested-by: Aaron Lu <aaron.lu@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'mm/page_alloc.c')
| -rw-r--r-- | mm/page_alloc.c | 17 |
1 files changed, 13 insertions, 4 deletions
diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 35c399d51530..63edba38ebc2 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1447,6 +1447,8 @@ static void free_pcppages_bulk(struct zone *zone, int count, struct per_cpu_pages *pcp) { int pindex = 0; + int min_pindex = 0; + int max_pindex = NR_PCP_LISTS - 1; int batch_free = 0; int nr_freed = 0; unsigned int order; @@ -1472,13 +1474,20 @@ static void free_pcppages_bulk(struct zone *zone, int count, */ do { batch_free++; - if (++pindex == NR_PCP_LISTS) - pindex = 0; + if (++pindex > max_pindex) + pindex = min_pindex; list = &pcp->lists[pindex]; - } while (list_empty(list)); + if (!list_empty(list)) + break; + + if (pindex == max_pindex) + max_pindex--; + if (pindex == min_pindex) + min_pindex++; + } while (1); /* This is the only non-empty list. Free them all. */ - if (batch_free == NR_PCP_LISTS) + if (batch_free >= max_pindex - min_pindex) batch_free = count; order = pindex_to_order(pindex); |