summaryrefslogtreecommitdiffstats
path: root/mm (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge tag 'slab-for-5.18' of ↵Linus Torvalds2022-03-233-80/+54
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab Pull slab updates from Vlastimil Babka: - A few non-trivial SLUB code cleanups, most notably a refactoring of deactivate_slab(). - A bunch of trivial changes, such as removal of unused parameters, making stuff static, and employing helper functions. * tag 'slab-for-5.18' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab: mm: slub: Delete useless parameter of alloc_slab_page() mm: slab: Delete unused SLAB_DEACTIVATED flag mm/slub: remove forced_order parameter in calculate_sizes mm/slub: refactor deactivate_slab() mm/slub: limit number of node partial slabs only in cache creation mm/slub: use helper macro __ATTR_XX_MODE for SLAB_ATTR(_RO) mm/slab_common: use helper function is_power_of_2() mm/slob: make kmem_cache_boot static
| * Merge branch 'slab/for-5.18/cleanups' into slab/for-linusVlastimil Babka2022-03-211-63/+42
| |\ | | | | | | | | | Non-trivial SLUB code cleanups, notably refactoring of deactivate_slab().
| | * mm/slub: refactor deactivate_slab()Hyeonggon Yoo2022-03-091-52/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplify deactivate_slab() by unlocking n->list_lock and retrying cmpxchg_double() when cmpxchg_double() fails, and perform add_{partial,full} only when it succeed. Releasing and taking n->list_lock again here is not harmful as SLUB avoids deactivating slabs as much as possible. [ vbabka@suse.cz: perform add_{partial,full} when cmpxchg_double() succeed. count deactivating full slabs even if debugging flag is not set. ] Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Roman Gushchin <roman.gushchin@linux.dev> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220307074057.902222-3-42.hyeyoo@gmail.com
| | * mm/slub: limit number of node partial slabs only in cache creationHyeonggon Yoo2022-03-091-11/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SLUB sets number of minimum partial slabs for node (min_partial) using set_min_partial(). SLUB holds at least min_partial slabs even if they're empty to avoid excessive use of page allocator. set_min_partial() limits value of min_partial limits value of min_partial MIN_PARTIAL and MAX_PARTIAL. As set_min_partial() can be called by min_partial_store() too, Only limit value of min_partial in kmem_cache_open() so that it can be changed to value that a user wants. [ rientjes@google.com: Fold set_min_partial() into its callers ] Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Roman Gushchin <roman.gushchin@linux.dev> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220307074057.902222-2-42.hyeyoo@gmail.com
| * | Merge branch 'slab/for-5.18/trivial' into slab/for-linusVlastimil Babka2022-03-213-17/+12
| |\ \ | | |/ | |/| | | | | | | | | | | | | Trivial slab code changes: - deleting unused parameters and flags - using helper macros and functions - making structures static
| | * mm: slub: Delete useless parameter of alloc_slab_page()Xiongwei Song2022-03-101-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The parameter @s is useless for alloc_slab_page(). It was added in 2014 by commit 5dfb41750992 ("sl[au]b: charge slabs to kmemcg explicitly"). The need for it was removed in 2020 by commit 1f3147b49d75 ("mm: slub: call account_slab_page() after slab page initialization"). Let's delete it. [willy@infradead.org: Added detailed history of @s] Signed-off-by: Xiongwei Song <sxwjean@gmail.com> Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org> Acked-by: David Rientjes <rientjes@google.com> Reviewed-by: Roman Gushchin <roman.gushchin@linux.dev> Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220310140701.87908-3-sxwjean@me.com
| | * mm/slub: remove forced_order parameter in calculate_sizesMiaohe Lin2022-03-091-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since commit 32a6f409b693 ("mm, slub: remove runtime allocation order changes"), forced_order is always -1. Remove this unneeded parameter to simplify the code. Signed-off-by: Miaohe Lin <linmiaohe@huawei.com> Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220309092036.50844-1-linmiaohe@huawei.com
| | * mm/slub: use helper macro __ATTR_XX_MODE for SLAB_ATTR(_RO)Lianjie Zhang2022-03-071-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows more concise code, and VERIFY_OCTAL_PERMISSIONS() can help validate any future change. Signed-off-by: Lianjie Zhang <zhanglianjie@uniontech.com> Acked-by: David Rientjes <rientjes@google.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220306073818.15089-1-zhanglianjie@uniontech.com
| | * mm/slab_common: use helper function is_power_of_2()Miaohe Lin2022-02-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Use helper function is_power_of_2() to check if KMALLOC_MIN_SIZE is power of two. Minor readability improvement. Signed-off-by: Miaohe Lin <linmiaohe@huawei.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220217091609.8214-1-linmiaohe@huawei.com
| | * mm/slob: make kmem_cache_boot staticMiaohe Lin2022-02-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | kmem_cache_boot is never accessed outside slob.c. Make it static. Signed-off-by: Miaohe Lin <linmiaohe@huawei.com> Acked-by: David Rientjes <rientjes@google.com> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Link: https://lore.kernel.org/r/20220217085842.29032-1-linmiaohe@huawei.com
* | | Merge tag 'folio-5.18b' of git://git.infradead.org/users/willy/pagecacheLinus Torvalds2022-03-239-59/+54
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull filesystem folio updates from Matthew Wilcox: "Primarily this series converts some of the address_space operations to take a folio instead of a page. Notably: - a_ops->is_partially_uptodate() takes a folio instead of a page and changes the type of the 'from' and 'count' arguments to make it obvious they're bytes. - a_ops->invalidatepage() becomes ->invalidate_folio() and has a similar type change. - a_ops->launder_page() becomes ->launder_folio() - a_ops->set_page_dirty() becomes ->dirty_folio() and adds the address_space as an argument. There are a couple of other misc changes up front that weren't worth separating into their own pull request" * tag 'folio-5.18b' of git://git.infradead.org/users/willy/pagecache: (53 commits) fs: Remove aops ->set_page_dirty fb_defio: Use noop_dirty_folio() fs: Convert __set_page_dirty_no_writeback to noop_dirty_folio fs: Convert __set_page_dirty_buffers to block_dirty_folio nilfs: Convert nilfs_set_page_dirty() to nilfs_dirty_folio() mm: Convert swap_set_page_dirty() to swap_dirty_folio() ubifs: Convert ubifs_set_page_dirty to ubifs_dirty_folio f2fs: Convert f2fs_set_node_page_dirty to f2fs_dirty_node_folio f2fs: Convert f2fs_set_data_page_dirty to f2fs_dirty_data_folio f2fs: Convert f2fs_set_meta_page_dirty to f2fs_dirty_meta_folio afs: Convert afs_dir_set_page_dirty() to afs_dir_dirty_folio() btrfs: Convert extent_range_redirty_for_io() to use folios fs: Convert trivial uses of __set_page_dirty_nobuffers to filemap_dirty_folio btrfs: Convert from set_page_dirty to dirty_folio fscache: Convert fscache_set_page_dirty() to fscache_dirty_folio() fs: Add aops->dirty_folio fs: Remove aops->launder_page orangefs: Convert launder_page to launder_folio nfs: Convert from launder_page to launder_folio fuse: Convert from launder_page to launder_folio ...
| * | | fs: Remove aops ->set_page_dirtyMatthew Wilcox (Oracle)2022-03-162-11/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With all implementations converted to ->dirty_folio, we can stop calling this fallback method and remove it entirely. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Convert __set_page_dirty_no_writeback to noop_dirty_folioMatthew Wilcox (Oracle)2022-03-164-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a mechanical change. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Convert __set_page_dirty_buffers to block_dirty_folioMatthew Wilcox (Oracle)2022-03-163-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Convert all callers; mostly this is just changing the aops to point at it, but a few implementations need a little more work. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | mm: Convert swap_set_page_dirty() to swap_dirty_folio()Matthew Wilcox (Oracle)2022-03-152-9/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Straightforward conversion. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Add aops->dirty_folioMatthew Wilcox (Oracle)2022-03-152-8/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This replaces ->set_page_dirty(). It returns a bool instead of an int and takes the address_space as a parameter instead of expecting the implementations to retrieve the address_space from the page. This is particularly important for filesystems which use FS_OPS for swap. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Add aops->launder_folioMatthew Wilcox (Oracle)2022-03-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since the only difference between ->launder_page and ->launder_folio is the type of the pointer, these can safely use a union without affecting bisectability. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Remove aops->invalidatepageMatthew Wilcox (Oracle)2022-03-151-11/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With all users migrated to ->invalidate_folio, remove the old operation. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Turn block_invalidatepage into block_invalidate_folioMatthew Wilcox (Oracle)2022-03-151-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove special-casing of a NULL invalidatepage, since there is no more block_invalidatepage. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Add invalidate_folio() aops methodMatthew Wilcox (Oracle)2022-03-151-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is used in preference to invalidatepage, if defined. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Turn do_invalidatepage() into folio_invalidate()Matthew Wilcox (Oracle)2022-03-152-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Take a folio instead of a page, fix the types of the offset & length, and export it to filesystems. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
| * | | fs: Convert is_partially_uptodate to foliosMatthew Wilcox (Oracle)2022-03-141-2/+2
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since the uptodate property is maintained on a per-folio basis, the is_partially_uptodate method should also take a folio. Fix the types at the same time so it's clear that it returns true/false and takes the count in bytes, not blocks. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Acked-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Tested-by: Mike Marshall <hubcap@omnibond.com> # orangefs Tested-by: David Howells <dhowells@redhat.com> # afs
* | | Merge tag 'folio-5.18c' of git://git.infradead.org/users/willy/pagecacheLinus Torvalds2022-03-2335-2571/+2370
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull folio updates from Matthew Wilcox: - Rewrite how munlock works to massively reduce the contention on i_mmap_rwsem (Hugh Dickins): https://lore.kernel.org/linux-mm/8e4356d-9622-a7f0-b2c-f116b5f2efea@google.com/ - Sort out the page refcount mess for ZONE_DEVICE pages (Christoph Hellwig): https://lore.kernel.org/linux-mm/20220210072828.2930359-1-hch@lst.de/ - Convert GUP to use folios and make pincount available for order-1 pages. (Matthew Wilcox) - Convert a few more truncation functions to use folios (Matthew Wilcox) - Convert page_vma_mapped_walk to use PFNs instead of pages (Matthew Wilcox) - Convert rmap_walk to use folios (Matthew Wilcox) - Convert most of shrink_page_list() to use a folio (Matthew Wilcox) - Add support for creating large folios in readahead (Matthew Wilcox) * tag 'folio-5.18c' of git://git.infradead.org/users/willy/pagecache: (114 commits) mm/damon: minor cleanup for damon_pa_young selftests/vm/transhuge-stress: Support file-backed PMD folios mm/filemap: Support VM_HUGEPAGE for file mappings mm/readahead: Switch to page_cache_ra_order mm/readahead: Align file mappings for non-DAX mm/readahead: Add large folio readahead mm: Support arbitrary THP sizes mm: Make large folios depend on THP mm: Fix READ_ONLY_THP warning mm/filemap: Allow large folios to be added to the page cache mm: Turn can_split_huge_page() into can_split_folio() mm/vmscan: Convert pageout() to take a folio mm/vmscan: Turn page_check_references() into folio_check_references() mm/vmscan: Account large folios correctly mm/vmscan: Optimise shrink_page_list for non-PMD-sized folios mm/vmscan: Free non-shmem folios without splitting them mm/rmap: Constify the rmap_walk_control argument mm/rmap: Convert rmap_walk() to take a folio mm: Turn page_anon_vma() into folio_anon_vma() mm/rmap: Turn page_lock_anon_vma_read() into folio_lock_anon_vma_read() ...
| * | | mm/damon: minor cleanup for damon_pa_youngMiaohe Lin2022-03-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | if need_lock is true but folio_trylock fails, we should return false instead of NULL to match the return value type exactly. No functional change intended. Signed-off-by: Miaohe Lin <linmiaohe@huawei.com> Reviewed-by: SeongJae Park <sj@kernel.org> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/filemap: Support VM_HUGEPAGE for file mappingsMatthew Wilcox (Oracle)2022-03-211-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the VM_HUGEPAGE flag is set, attempt to allocate PMD-sized folios during readahead, even if we have no history of readahead being successful. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/readahead: Switch to page_cache_ra_orderMatthew Wilcox (Oracle)2022-03-213-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | do_page_cache_ra() was being exposed for the benefit of do_sync_mmap_readahead(). Switch it over to page_cache_ra_order() partly because it's a better interface but mostly for the benefit of the next patch. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/readahead: Align file mappings for non-DAXWilliam Kucharski2022-03-211-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we have the opportunity to use PMDs to map a file, we want to follow the same rules as DAX. Signed-off-by: William Kucharski <william.kucharski@oracle.com> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/readahead: Add large folio readaheadMatthew Wilcox (Oracle)2022-03-211-7/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allocate large folios in the readahead code when the filesystem supports them and it seems worth doing. The heuristic for choosing which folio sizes will surely need some tuning, but this aggressive ramp-up has been good for testing. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/filemap: Allow large folios to be added to the page cacheMatthew Wilcox (Oracle)2022-03-211-17/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We return -EEXIST if there are any non-shadow entries in the page cache in the range covered by the folio. If there are multiple shadow entries in the range, we set *shadowp to one of them (currently the one at the highest index). If that turns out to be the wrong answer, we can implement something more complex. This is mostly modelled after the equivalent function in the shmem code. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm: Turn can_split_huge_page() into can_split_folio()Matthew Wilcox (Oracle)2022-03-212-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This function already required a head page to be passed, so this just adds type-safety and removes a few implicit calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/vmscan: Convert pageout() to take a folioMatthew Wilcox (Oracle)2022-03-211-32/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We always write out an entire folio at once. This conversion removes a few calls to compound_head() and gets the NR_VMSCAN_WRITE statistic right when writing out a large folio. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/vmscan: Turn page_check_references() into folio_check_references()Matthew Wilcox (Oracle)2022-03-211-17/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This function only has one caller, and it already has a folio. This removes a number of calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/vmscan: Account large folios correctlyMatthew Wilcox (Oracle)2022-03-211-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The statistics we gather should count the number of pages, not the number of folios. The logic in this function is somewhat convoluted, but even if we split the folio, I think the accounting is now correct. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/vmscan: Optimise shrink_page_list for non-PMD-sized foliosMatthew Wilcox (Oracle)2022-03-211-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | A large folio which is smaller than a PMD does not need to do the extra work in try_to_unmap() of trying to split a PMD entry. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/vmscan: Free non-shmem folios without splitting themMatthew Wilcox (Oracle)2022-03-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have to allocate memory in order to split a file-backed folio, so it's not a good idea to split them in the memory freeing path. It also doesn't work for XFS because pages have an extra reference count from page_has_private() and split_huge_page() expects that reference to have already been removed. Unfortunately, we still have to split shmem THPs because we can't handle swapping out an entire THP yet. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Constify the rmap_walk_control argumentMatthew Wilcox (Oracle)2022-03-213-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rmap walking functions do not modify the rmap_walk_control, and page_idle_clear_pte_refs() takes advantage of that to move construction of the rmap_walk_control to compile time. This lets us remove an unclean cast. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Convert rmap_walk() to take a folioMatthew Wilcox (Oracle)2022-03-217-91/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This ripples all the way through to every calling and called function from rmap. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm: Turn page_anon_vma() into folio_anon_vma()Matthew Wilcox (Oracle)2022-03-214-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the prototype from mm.h to mm/internal.h and convert all callers to pass a folio. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Turn page_lock_anon_vma_read() into folio_lock_anon_vma_read()Matthew Wilcox (Oracle)2022-03-213-7/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add back page_lock_anon_vma_read() as a wrapper. This saves a few calls to compound_head(). If any callers were passing a tail page before, this would have failed to lock the anon VMA as page->mapping is not valid for tail pages. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/damon: Convert damon_pa_young() to use a folioMatthew Wilcox (Oracle)2022-03-211-12/+15
| | | | | | | | | | | | | | | | | | | | | | | | Ensure that we're passing the entire folio to rmap_walk(). Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/damon: Convert damon_pa_mkold() to use a folioMatthew Wilcox (Oracle)2022-03-211-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | Ensure that we're passing the entire folio to rmap_walk(). Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/migrate: Convert remove_migration_ptes() to foliosMatthew Wilcox (Oracle)2022-03-213-41/+53
| | | | | | | | | | | | | | | | | | | | | | | | Convert the implementation and all callers. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Convert make_device_exclusive_range() to use foliosMatthew Wilcox (Oracle)2022-03-211-27/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the PageTail check earlier so we can avoid even taking the folio lock on tail pages. Otherwise, this is a straightforward use of folios throughout. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Convert try_to_migrate() to foliosMatthew Wilcox (Oracle)2022-03-214-34/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Convert the callers to pass a folio and the try_to_migrate_one() worker to use a folio throughout. Fixes an assumption that a folio must be <= PMD size. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Convert try_to_unmap() to take a folioMatthew Wilcox (Oracle)2022-03-216-51/+60
| | | | | | | | | | | | | | | | | | | | | | | | Change all three callers and the worker function try_to_unmap_one(). Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/huge_memory: Convert __split_huge_pmd() to take a folioMatthew Wilcox (Oracle)2022-03-212-25/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Convert split_huge_pmd_address() at the same time since it only passes the folio through, and its two callers already have a folio on hand. Removes numerous calls to compound_head() and removes an assumption that a page cannot be larger than a PMD. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Turn page_referenced() into folio_referenced()Matthew Wilcox (Oracle)2022-03-213-44/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both its callers pass a page which was previously on an LRU list, so were passing a folio by definition. Use the type system to enforce that and remove a few calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Christoph Hellwig <hch@lst.de>
| * | | mm/mlock: Add mlock_vma_folio()Matthew Wilcox (Oracle)2022-03-212-15/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Convert mlock_page() into mlock_folio() and convert the callers. Keep mlock_vma_page() as a wrapper. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
| * | | mm/rmap: Use a folio in page_mkclean_one()Matthew Wilcox (Oracle)2022-03-211-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | folio_mkclean() already passes down a head page, so convert it back to a folio. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Christoph Hellwig <hch@lst.de>
| * | | mm/page_idle: Convert page_idle_clear_pte_refs() to use a folioMatthew Wilcox (Oracle)2022-03-211-9/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The PG_idle and PG_young bits are ignored if they're set on tail pages, so ensure we're passing a folio around. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Christoph Hellwig <hch@lst.de>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-30 08:26:06 +0100